Rodney Brooks: Robotics | Lex Fridman Podcast #217

Lex Fridman 0:00 The following is a conversation with Rodney Brooks, one of the greatest roboticists in history. He led the Computer Science and Artificial Intelligence Laboratory at MIT, then co-founded iRobot, which is one of the most successful robotics companies ever. Then, he co-founded Rethink Robotics that created some amazing collaborative robots, like Baxter and Sawyer. Finally, he co-founded Robust.AI, whose mission is to teach robots common sense, which is a lot harder than it sounds. To support this podcast, please check out our sponsors in the description. As a side note, let me say that Rodney is someone I've looked up to for many years in my now over two decade journey in robotics because, one, he's a legit great engineer of real world systems, and two, he's not afraid to state controversial opinions that challenge the way we see the AI world. But of course, while I agree with him on some of his critical views of AI, I don't agree with some others, and he's fully supportive of such disagreement. Nobody ever built anything great by being fully agreeable. There's always respect and love behind our interactions, and when a conversation is recorded like it was for this podcast, I think a little bit of disagreement is fun. This is the Lex Fridman Podcast, and here is my conversation with Rodney Brooks.

Lex Fridman 1:31 What is the most amazing or beautiful robot that you've ever had the chance to work with?

Rodney Brooks 1:37 I think it was DOMO, um, which was made by one of my grad students, Aaron Edsinger. It now sits in, uh, Daniela Rus' office, uh, director of CSAIL, and it was just a beautiful robot. And Aaron was really clever. He didn't give me a budget ahead of time. He didn't tell me what he was gonna do. He just started spending money.

Lex Fridman 1:58 (laughs)

Rodney Brooks 1:58 He spent a lot of money.

Lex Fridman 1:59 Yeah.

Rodney Brooks 2:00 He and Jeff Weber, who, um, is a mechanical engineer, who Aaron insisted he bring with him when he became a grad student-

Lex Fridman 2:07 Mm-hmm.

Rodney Brooks 2:07 ... built this beautiful, gorgeous robot, DOMO, which is a to- upper torso, humanoid, two, two arms, uh, uh-

Lex Fridman 2:16 With fingers?

Rodney Brooks 2:16 ... three-fingered hands-

Lex Fridman 2:17 Mm.

Rodney Brooks 2:17 ... um, uh, and, um, a face, eyeballs, um, all, uh, not the, not the eyeballs, but everything else, series elastic actuators, uh, you can interact with it, um, cable driven, all the motors are inside, and it's just gorgeous.

Lex Fridman 2:34 The eyeballs are actuated too or no?

Rodney Brooks 2:35 Oh yeah, the eyeballs are actuated with cameras, and, you know, so it had a visual attention mechanism, you know-

Lex Fridman 2:40 Wow.

Rodney Brooks 2:41 ... looking when people came in and looking at their face and talking with them. Yeah.

Lex Fridman 2:46 Why was it amazing?

Rodney Brooks 2:48 The beauty of it. You said bea- but you said what was the most beau-

Lex Fridman 2:50 I said beauty. What is the most beautiful?

Rodney Brooks 2:52 It's just mechanically gorgeous, as, as everything Aaron builds has always been mechanically gorgeous. It's just exquisite in the detail.

Lex Fridman 3:00 Uh, we're talking about mechanically, like, literally the amount of actuators, the, uh-

Rodney Brooks 3:04 The actuators, the cables, he, um, anodizes different parts different colors, and it just looks like a work of art.

Lex Fridman 3:13 What about the face? Is it, do you find the face beautiful in robots?

Rodney Brooks 3:17 Um, when you make a, a robot, uh, it's making a promise for how well it will be able, able to interact, so I always encourage my students not to overpromise, you know, and-

Lex Fridman 3:27 Like, even with its essence, like, the w- the thing it presents, it should not overpromise?

Rodney Brooks 3:32 Yeah, so I... The, the joke I make, which I think you'll get, is if your robot looks like Albert Einstein, it should be as smart as Albert Einstein.

Lex Fridman 3:39 Yeah.

Rodney Brooks 3:39 So, uh, the, the only thing in, in DOMO's face is, uh, the eyeballs, um, and 'cause that's all it can do. It can look at you and pay attention, um, and so there is no f- it's not like one of those, um, Japanese robots that looks exactly like a person at all.

Lex Fridman 3:59 But see, the thing is, us humans, and dogs too, don't just use eyes for, as attentional mechanisms. They also use it to communicate. It's part of the communication. Like a dog can look at you, look at another thing, and look back at you, and that designates that we're going to be looking at that thing together.

Rodney Brooks 4:16 Yeah, or, or intent, you know?

Lex Fridman 4:17 Intent.

Rodney Brooks 4:18 And on, on, on both Baxter and Sawyer at Rethink Robotics, they had a screen with, you know, graphic eyes, so it wasn't actually where the cameras were pointing, but it... The, the eyes would look in the direction it was about to move its arms, so people in the factory nearby were not surprised by its motions 'cause it gave that intent away.

Lex Fridman 4:40 Before we talk about Baxter, which I think is a beautiful robot, let's go back to the beginning. When did you first fall in love with robotics? We're talking about beauty and love to open the conversation. This is great.

Rodney Brooks 4:53 I've got these... I was born in, uh, the end of 1954 in... I grew up in Adelaide, South Australia, and I have these two books that are dated 1961, so I'm guessing my mother found them in a store in '62 or '63, How and Why Wonder Books. Um, How and Why Wonder Book of Electricity and a How and Why Wonder Book of Giant Brains and Robots. (laughs)

Lex Fridman 5:18 (laughs)

Rodney Brooks 5:18 And, uh, (laughs) um, I learned how to build circuits, you know, when I was eight or nine, simple circuits, and I, and I read, you know, learned the binary system, and, um, saw all these drawings, mostly, uh, of robots, and, um, then I tried to build them for the rest of my childhood.

Lex Fridman 5:38 Wait. '61 you said?

Rodney Brooks 5:40 This was when the, the two books. I've still got them at home.

Lex Fridman 5:43 What, what does the robot mean in that context?

Rodney Brooks 5:45 No, they, they were... Uh, some of the robots, uh, that they had were, uh, arms, you know, big arms to move nuclear material around, but they had pictures of welding robots that looked like humans under the sea welding stuff underwater, um, so they weren't real robots. Uh, but they were, you know, what people were thinking about for robots.

Lex Fridman 6:07 What were you thinking about? Were you thinking about humanoids? Were you thinking about arms with fingers? Were you thinking about faces or cars or-

Rodney Brooks 6:14 No, actually, to be honest, I realized my limitation on building mechanical stuff, so I just built, um, the brains f- mostly. I, I, out of different technologies as I got older. Um, uh, I, I built a, uh, a learning system, which, uh, was chemical-based, and I had this ice cube tray. Each well was a cell and, uh, by applying voltage to the two electrodes, uh, it would build up a copper bridge. So over time, it would, it would learn, uh, s- a, a simple network, um, so I could teach it stuff. And that was... Mostly things were driven by my budget, and nails as electrodes and a, and a, and a ice cream, I mean, a ice cube tray was, was about my budget at that stage. Later I managed to buy transistors and then I could build gates and flip-flops and stuff.

Lex Fridman 7:07 So, so one, one of your first robots wa- was an ice cube tray?

Rodney Brooks 7:11 Yeah. (laughs)

Lex Fridman 7:12 (laughs)

Rodney Brooks 7:13 And it was very cerebral because it learned to add. (laughs)

Lex Fridman 7:15 (laughs) Okay. Very nice. Uh, well, just a, a decade or so before in, in, uh, 1950, Alan Turing wrote the paper that formulated the Turing test, and he opened that paper with the question, "Can machines think?" So let me ask you this question. Can machines think? Can your ice cube tray one day think?

Rodney Brooks 7:40 Um, certainly machines can think because I believe you're a machine and I'm a machine and I believe we both think. Um, I think-

Lex Fridman 7:47 Speak for yourself.

Rodney Brooks 7:47 ... I, I think any other (laughs) philosophical position is sort of a little ludicrous. What does think mean if, if it's not something that we do? Um, and we, and we are machines. Um, so, yes, machines can, but do we have a clue how to build such machines? That's a very different question. Are we capable of building such machines? Uh, you know, are we smart enough? We think we're smart enough to do anything, but maybe we're not. Maybe, you know, we're just not s- smart enough to build stuff like us.

Lex Fridman 8:15 The kind of computer that, uh, Alan Turing was thinking about, do you think there is something fundamentally, uh, or significantly different between the computer between our ears, the, the biological computer that humans use, and, uh, the computer that he was thinking about from a, from a sort of high level, philosophical?

Rodney Brooks 8:35 Yeah, I be- I believe that, uh, that it's very wrong. In fact, I'm halfway through a, I think it'll be about a 480-page book, um, titled, the working title is Not Even One Wrong.

Lex Fridman 8:48 (laughs)

Rodney Brooks 8:48 And if I may, I will tell you a bit about that book.

Lex Fridman 8:50 Yes, please.

Rodney Brooks 8:51 So there's two, two, well, three thrusts to it. Um, one is the history of computation, what we call computation.

Lex Fridman 8:58 Mm-hmm.

Rodney Brooks 8:59 Goes all the way back to, uh, uh, some manuscripts in Latin from 1614 and 1620 by Napier and, and Kepler, through Babbage and, and Lovelace. And then, Turing's 1936 paper, uh, is wha- you know, what we, what we think of as the invention of, of, of modern computation. And that paper, uh, by the way, did not set out to, you know, invent computation. It set out to, uh, negatively answer one of, uh, Hilbert's three later set of problems. He called it, um, uh, effe- uh, is an effective way of, of, of, of getting answers. And, and Hilbert, Hilbert really worked with rewriting rules, as did, um, um, Church, who also, at the same time, a month earlier than Turing, disproved Hilbert's, one of these three hypotheses. The other two had b- already been disproved by Godel. So Turing set out to disprove it because that's, it's always easier to disprove these things than to prove that there is an answer. And, um, so he needed, um... And it, it really came from his, his, uh, professor while he was an undergrad at, at Cambridge who'd said, who'd turned it into, is there a mechanical process? So he wanted to have a, show a mechanical process that could calculate numbers because that was a mechanical process that people used to generate tables. They were called computers, the people at the time.

Lex Fridman 10:30 Mm-hmm.

Rodney Brooks 10:31 And they followed a set of rules where they had paper and they would write numbers down, and based on the numbers, they would keep writing other numbers. And they would produce, um, numbers for these tables, engineering tables, that the more n- the more iterations they did, the more significant digits came out. And so Turing, in that paper, set out to sh- to, to find what sort of machine could do that, mechanical machine, where it could produce an arbitrary number of digits in the same way a human computer did. Um, and he came up with a very simple set of constraints where there was an, an infinite supply of paper. This was the, the tape of the Turing machine. And each Turing machine had a s- set of in- came with a set of instructions that, as a person could do with pencil and paper, write down things on the tape and erase them and put new things there. And he was able to show that that system was not able to do something that Hilbert had hypothesized. So he disproved it. But he had to show that this was, this, this system was good enough to do whatever could be done, but couldn't do this other thing.

Lex Fridman 11:48 (laughs) Yeah.

Rodney Brooks 11:48 And there he said, and he says in the paper, "I don't have any real arguments for this, but based on intuition..." So that's how he defined computation. And then if you look over the next... Uh, from 1936 up until really around 1975, you see people struggling with w-... "Is this really what computation is?"

Lex Fridman 12:09 Mm-hmm.

Rodney Brooks 12:10 And so, Marvin Minsky, very well known in AI but also a fantastic mathematician, uh, in his book Finite and Infinite Machines from the mid '60s, which is a beautiful, beautiful mathematical book, um, ha- uh, says- says at the start of the book, "Well, what is computation? Turing says it's this and, yeah, I sort of think it's that. It doesn't really matter whether the stuff's made of wood or plastic. It's just, you know, that relatively cheap stuff can do this stuff." And so, yeah, seems like computation. And Donald Knuth, uh, in his first volume of his, you know, Uh, Art of Computer Programming-

Lex Fridman 12:46 Mm-hmm.

Rodney Brooks 12:47 ... in around 1968 says, "Well, what's computation? Uh, it's just stuff, like Turing says, that a person could do each step without too much trouble."

Lex Fridman 12:58 (laughs)

Rodney Brooks 12:58 Uh, (laughs) and so one of his examples-

Lex Fridman 13:00 (laughs) .

Rodney Brooks 13:00 ... of what would be too much trouble was, uh, a step which required knowing, um, whether Fermat's Last Theorem was true or not, because it was not known at the time. And that's too much trouble for a person to do as a step. And, um, Hopcroft and Ullman sort of said a similar thing later that year. And by 1975 in the Aho, Hopcroft, and Ullman book, they're saying, "Well, you know, we don't really know what computation is, but intuition says this is sort of about right, and this is what it is." That's computation. It's a sort of agreed upon thing which happens to be really easy to implement in silicon. And then we had Moore's Law, which took off, and it's been an incredibly powerful tool.

Lex Fridman 13:45 Mm-hmm.

Rodney Brooks 13:46 I certainly wouldn't argue with that. The version we have of computation, incredibly powerful.

Lex Fridman 13:51 C- can we just g- uh, take a pause? So, what we're talking about is there's an infinite tape with some simple rules on how to write and- on that tape, and that's- that's what we're kind of thinking about. This is computation.

Rodney Brooks 14:01 Yeah, and it's modeled after humans, how humans do stuff, and- and- uh, w- I think it's, uh, Turing says in the '36 paper, "One of the critical facts here is that a human has a limited amount of memory."

Lex Fridman 14:12 Mm-hmm.

Rodney Brooks 14:13 "So that's what we're gonna put onto our comp- our mechanical computers."

Lex Fridman 14:16 Mm-hmm (laughs) .

Rodney Brooks 14:19 So, mm, mm, mm. So, you know, unlike mass-

Lex Fridman 14:21 (laughs)

Rodney Brooks 14:21 ... unlike mass or charge or, you know-

Lex Fridman 14:23 Yeah, the memory.

Rodney Brooks 14:23 ... it's not a- it's- it's not given by the universe. It was, "This is what we're gonna call computation."

Lex Fridman 14:28 Yeah.

Rodney Brooks 14:29 And then it has this really, uh, you know, it had this really good implementation, which has completely changed our technological world. That's computation. S- second part of the book, I, or- or- or- or argument of the book, I have this two by two matrix with, um, science, uh, in the top row, engineering in the bottom row, left column is intelligence, right column is life.

Lex Fridman 14:57 Mm-hmm.

Rodney Brooks 14:58 So, in the bottom row, the engineering, there's artificial intelligence and there's artificial life.

Lex Fridman 15:03 Mm-hmm.

Rodney Brooks 15:03 In the top row, there's neuroscience and abiogenesis. How does living matter turn in... How does non-living matter become living matter?

Lex Fridman 15:12 Yes.

Rodney Brooks 15:12 Four disciplines. These four disciplines all, um, came into their current form in the period 1945 to 1965. Um...

Lex Fridman 15:23 (laughs) That's interesting.

Rodney Brooks 15:25 There was neuroscience before, but it wa- it wasn't effective neuroscience.

Lex Fridman 15:28 Yeah.

Rodney Brooks 15:28 It was, you know, there were these ganglia and those electrical charges, but no one knews- knows what to do with it. And furthermore, there al- there were a lot of players who are common across them. Uh, I- I've got, I've- I've identified common players, except for artificial intelligence and abiogenesis, I don't have. But for any other pair, I can point to people who worked them. And a whole bunch of them, by the way, were at the Research Lab for Electronics at MIT-

Lex Fridman 15:53 Mm-hmm.

Rodney Brooks 15:53 ... um, where, uh, Warren McCulloch, uh, held- held- held forth. And, in fact, uh, McCulloch, uh, Pitts, uh, Lettvin, and Maturana wrote the first paper on functional neuroscience called What the Frog's Eye Tells the Frog's Brain, where instead of it just being this bunch of nerves, they sort of showed what different anatomical components were doing and telling other anatomical components and, m- you know, generating behavior in the frog.

Lex Fridman 16:24 And w- would you put them as basically the fathers, the, or the, one of the early pioneers of what are now called artificial neural networks?

Rodney Brooks 16:33 Y- yeah. I mean, McCulloch and Pitts, uh, uh, Pitts was, uh, was, uh, much younger than him, in 1943-

Lex Fridman 16:40 Mm-hmm.

Rodney Brooks 16:40 ... had written a paper inspired by Bertrand Russell on A Calculus for the Ideas Eminent in Neural Systems.

Lex Fridman 16:51 Mm-hmm.

Rodney Brooks 16:51 Where they had tried to, without any real proof, they had tried to give a formalism for neurons basically in terms of logic. AND gates, OR gates, and NOT gates, with- with no real evidence that that was what was going on. But they- they talked about it, and that- that was picked up by Minsky for his 1953, uh, dissertation on, uh, which was a- was a neural network, we would call it today. It was picked up by, um, uh, John von Neumann when he was designing the EDVAC computer in 1945, he talked about its components being neurons, um, based on-

Lex Fridman 17:30 Mm-hmm.

Rodney Brooks 17:30 ... and in references. He's only got three references, and one of them is the McCulloch-Pitts paper. So, all these people and then the AI people and the artificial life people, which was John von Neumann originally, and-

Lex Fridman 17:42 There's like overlap between all of them.

Rodney Brooks 17:43 They're all going around the same time. And three of these four disciplines turned to computation as their primary metaphor.

Lex Fridman 17:51 Mm-hmm.

Rodney Brooks 17:52 So, I- I- I've got a couple of chapters in the book. One is titled, "Wait, Computers are People?" 'Cause that's where our computers came from-

Lex Fridman 18:01 Yeah.

Rodney Brooks 18:02 ... (laughs) and, uh, you know, from people who were computing stuff. And then I've got another chapter, "Wait, People are Computers?"

Lex Fridman 18:09 (laughs)

Rodney Brooks 18:09 Which is about computational neuroscience.

Lex Fridman 18:11 Yes.

Rodney Brooks 18:11 So there's this whole circle here.... and that, that computation is it. And, you know, I have talked to, to people about, "Well, maybe it's not computation that goes on in the head." Of course, it is.

Lex Fridman 18:24 Yeah.

Rodney Brooks 18:24 Okay, well, when Elon Musk's rocket goes up, is it computing? Is that how it gets into orbit, by computing? But we've got this idea if you wanna build an AI system, you write a computer program.

Lex Fridman 18:40 Yeah, in a sense... So, the word computation very quickly starts doing a lot of work that was not initially intended to, to, to do. It's the same, if you talk about the universe as essentially performing a computation.

Rodney Brooks 18:53 Yeah, right. W- Wolfram does this.

Lex Fridman 18:55 Yeah.

Rodney Brooks 18:55 He turns it into computation. You don't turn rockets into computation.

Lex Fridman 18:59 Yeah. By the way, when you say computation in our conversation, do you tend to think of computation narrowly in the way Turing thought of computation?

Rodney Brooks 19:08 I- it- it- it's, it's, it's gotten very...

Lex Fridman 19:10 Okay.

Rodney Brooks 19:11 It... You know-

Lex Fridman 19:13 (laughs)

Rodney Brooks 19:13 ... squishy.

Lex Fridman 19:14 Yeah.

Rodney Brooks 19:14 Squishy.

Lex Fridman 19:15 Okay.

Rodney Brooks 19:15 Over time. Um, but computation and the way Turing thinks about it and the way most people think about it, actually fits very well with, um, thinking like a hunter-gatherer. There are places, and there can be stuff in places, and the stuff in places can change, and it stays there until someone changes it. And it's this m- m- metaphor of place and container which, you know, is a combination of our, um, place cells in our hippocampus and, uh, cortex. But this is, this is how we use metaphors for mostly to think about. When we get outside of our metaphor range, we have to invent tools which we can sort of switch on to use. So calculus is an example of a tool.

Lex Fridman 20:02 Mm-hmm.

Rodney Brooks 20:02 It can do stuff that our raw reasoning can't do, and we've got conventions of when you can use it or not. But sometimes, um, you know, people try to... O- all the time, we always try to get physical metaphors for things, which is why quantum mechanics has been such a problem for a hundred years, because, "It's a particle. No, it's a wave. It's got to be something we understand." And I say, "No, it's some weird mathematical object that's different from those." But we want that metaphor. Well, you know, I, I suspect that, that, you know, a hundred years or two hundred years from now, neither quantum mechanics nor, nor dark matter will be talked about in the same terms, you know, in the same way that, um, Phlogiston's theory eventually went away, um, uh, 'cause it, it just wasn't an adequate explanatory metaphor, you know? That metaphor was the stuff, there is stuff in the burning, the burning is in the matter.

Lex Fridman 20:58 (laughs)

Rodney Brooks 20:58 But as it turns out, the burning was outside the matter, it was the oxygen.

Lex Fridman 21:01 So, our desire for metaphor and combined with our limited cognitive capabilities gets us into trouble.

Rodney Brooks 21:08 That's my argument in this book. Now, and then people say, "Well, what is it then?" And I say, "Well, I wish I knew that and write a book about that." (laughs)

Lex Fridman 21:14 Yeah.

Rodney Brooks 21:14 But I, you know, give some ideas but... So, so there's this, there are three things. Computation is sort of a particular thing we use, um, uh... Oh, can I tell you one beautiful thing, one beautiful thing I found?

Lex Fridman 21:27 Yes, please.

Rodney Brooks 21:27 So, you know, I used an example of a thing that's different from computation. You hit a drum and it vibrates, and there are some, some, uh, stationary points on the drum's surface, you know, 'cause the waves are going up and down the stationary points. Um, now, uh, you could compute them to arbitrary precision, um, but the drum just knows them, the drum doesn't have to compute. What was the very first computer program ever written by Ada Lovelace?

Lex Fridman 21:54 Mm-hmm.

Rodney Brooks 21:55 To compute Bernoulli numbers, and Bernoulli numbers are exactly what you need to find those stable points in the drum's surface.

Lex Fridman 22:01 Wow.

Rodney Brooks 22:02 Anyway... (laughs) And there was a bug in her program, uh... (laughs)

Lex Fridman 22:06 (laughs)

Rodney Brooks 22:06 The arguments to divide were reversed in one place. (laughs)

Lex Fridman 22:10 And it still worked?

Rodney Brooks 22:11 Well, no, she, she never got to run it.

Lex Fridman 22:12 Oh, right. Okay.

Rodney Brooks 22:12 They never built the analytical engine, she wrote the program without, without it, you know. (laughs)

Lex Fridman 22:18 (laughs) Uh, so, uh, so computation?

Rodney Brooks 22:21 Computation is sort of, you know, a thing that's become dominant as a metaphor, but-

Lex Fridman 22:26 Yeah.

Rodney Brooks 22:26 ... eh, is it the right metaphor? Um, all three of these four fields adopted computation and, you know, the... A- a lot of it swirls around Warren McCulloch and his, all his students, and he founded a lot of people. Um, and, and, uh, and, and our human metaphors, our limitations to human thinking will play into this.

Lex Fridman 22:50 Yeah.

Rodney Brooks 22:50 They're the three themes of the book.

Lex Fridman 22:52 So-

Rodney Brooks 22:53 So, I have a little to say about computation. (laughs)

Lex Fridman 22:56 (laughs) So, uh, s- you're saying that there is a gap between the computer or the, the, um, the machine that performs computation, and this machine that appears to have consciousness and intelligence?

Rodney Brooks 23:13 Yeah.

Lex Fridman 23:14 Can we, um-

Rodney Brooks 23:14 That piece of meat in your head.

Lex Fridman 23:16 Piece of meat.

Rodney Brooks 23:16 And maybe it's not just the, the meat in your head, it's the rest of you, too. I mean, you have, you have, you actually have a neural system in your gut, um.

Lex Fridman 23:25 I, I tend to also believe... Not believe, but we're now dancing around things we don't know. But I tend to believe other humans are important. Like, so, we are almost like... I, I, I just don't think we would ever have achieved the level of intelligence we have without other humans. Uh, uh, I'm not saying so confidently, but I have an intuition that some of the intelligence is in the interaction.

Rodney Brooks 23:51 Yeah, and, and I think, uh, you know, I think it, uh, it seems to be very likely, again, we- you know, this is speculation, but we, our species and probably, um, probably Neanderthals to some extent, because you can find, uh, old bones where they seem to be counting on them by putting notches, um, that were Neothan- that Nean- Neanderthals had done. We are able to put, um, some of our stuff outside our body into the world.... and then other people can share it.

Lex Fridman 24:20 Mm-hmm.

Rodney Brooks 24:20 And then we get these tools that become shared tools. And so there's a whole coupling that would not occur in, you know, the single deep learning network which was fed, you know, all of the literature or something.

Lex Fridman 24:33 (laughs) Yeah. The, the, the neural network can't, um, step outside of itself. But is there, is there some, um ... Can we explore this dark room, uh, a little bit and try to get at something? Wh- wh- what is the magic? Where does the magic come from in the human brain that creates the mind? What, what, what's your sense as, as scientists that try to, uh, understand it and try to build it? What are the directions that, um, if followed might be productive? Is it creative interactive robots? Is it creating large deep neural networks that, uh, do, like, self-supervised learning and just, like, will, will, will discover that when you make something large enough some interesting things will emerge? Is it through physics and chemistry and biology, like artificial life angle? Like, we'll sneak up in this, uh, four quadrant matrix that you mentioned. Is there anything y- you're most-

Rodney Brooks 25:31 Yeah.

Lex Fridman 25:31 ... if you had to bet all your money (laughs) -

Rodney Brooks 25:33 (laughs) .

Lex Fridman 25:33 ... financial advice-

Rodney Brooks 25:34 I wouldn't.

Lex Fridman 25:35 (laughs) Okay.

Rodney Brooks 25:36 So, every intelligence we know, and that includes, you know, animal intelligence, dog intelligence, you know, um, um, octopus intelligence, which is a very different sort of architecture from, from us. Um, all the intelligences we know, um, perceive the world in some way, um, and then have action in the world, but they're able to, um, perceive objects in a way which is actually pretty damn phenom- phenomenal (laughs) and surprising. You know, we tend to think, you know, that, um, that, that, uh, the box over here between us, which is a sound box I think-

Lex Fridman 26:21 Mm-hmm.

Rodney Brooks 26:21 ... is what, is a blue box. But blueness, um, is something that we construct with, with, um, um, well, uh, color constancy. It's not a, it's not a, it's not... The blueness is not a direct function of the photons we're receiving, it's actually context, you know, which is why, um, y- you can turn, uh, you know, you, you maybe seen the examples where, um, someone turns a stop sign into a, um, some other sort of sign by just putting a couple of marks on them-

Lex Fridman 26:53 Mm-hmm.

Rodney Brooks 26:53 ... and the deep learning system gets it wrong. And everyone says, "But the stop sign's red. It, you know, why does it-"

Lex Fridman 26:58 Yeah.

Rodney Brooks 26:58 "... why does it think it's the other sort of sign?" Because redness is not intrinsic in just the photons, it's actually a construction of an understanding of the whole world and the relationship between objects to get con- color constancy. Um, but our, uh, tendency in order that we get an archive paper really quickly is you just show a lot of data and g- give-

Lex Fridman 27:17 Yeah.

Rodney Brooks 27:17 ... the labels and hope it figures it out. But it's not figuring it out in the same way we do. We have a very complex perceptual understanding of the world. Dogs have a very different perceptual understanding based on smell. They go smell, smell a post, they can tell, um, how many, y- you know, different dogs have visited it in the last 10 hours and how long ago. There's all sorts of stuff that we just don't perceive about the world. And just taking a single snapshot is not perceiving about the world. It's not perceiving the, the registration between us and the object. And registration is a, a philosophical concept. Brian Cantwell Smith talks about it a lot, very difficult squirmy thing to understand.

Lex Fridman 27:59 Mm-hmm.

Rodney Brooks 27:59 But I think none of our systems do that. We've always talked ab- in AI about the symbol grounding problem, how symbols that we talk about are grounded in the world.

Lex Fridman 28:07 Mm-hmm.

Rodney Brooks 28:08 And when deep learning came along and started labeling images, people said, "Ha, the grounding problem has been solved." No, the labeling problem was solved with some percentage accuracy, which is different from the grounding problem.

Lex Fridman 28:20 So you, uh, there's a, you agree with Hans Moravec and, uh, what's called the Moravec's paradox that highlights this counterintuitive notion that reasoning is easy, but perception i- and mobility are hard.

Rodney Brooks 28:39 Yeah. We shared an office when, um, we were, when I was working on computer vision and he was working on his, his first mobile robot.

Lex Fridman 28:46 What were those conversations like?

Rodney Brooks 28:48 They were great. (laughs)

Lex Fridman 28:50 (laughs) So do you still kind of, maybe you can elaborate, and do you still believe this kind of notion that perception is really, uh, hard and, like, can you make sense of why we humans have this poor intuition about what's hard and not in AI?

Rodney Brooks 29:04 Well, well, well, let me, let me give a sort of a, a, an- another, another story.

Lex Fridman 29:10 Sure.

Rodney Brooks 29:11 If you go back to, you know, the original, um, you know, teams working on AI, um, from the late '50s into the '60s, you know, and you go to the AI lab at MIT, um, who was it that was doing that? Was a bunch of really smart kids who got into MIT.

Lex Fridman 29:31 Mm-hmm.

Rodney Brooks 29:31 And they were intelligent. So what's intelligence about? Well, the stuff they were good at, playing chess, doing integrals. That was, that was hard stuff.

Lex Fridman 29:41 Yeah.

Rodney Brooks 29:41 But, you know, a baby could see stuff. That wasn't, that wasn't intelligence. That was... Any- anyone could do that, that's not intelligence.

Lex Fridman 29:47 Yeah.

Rodney Brooks 29:48 And so it, you know, this, there was this intuition that the hard stuff is the things they were good at and the easy stuff was the stuff that everyone could do.

Lex Fridman 29:58 Yeah.

Rodney Brooks 29:58 And maybe I'm overplaying it a little bit, but I think there's an element of that.

Lex Fridman 30:02 Yeah. I mean, there... I don't know how much truth there is to, uh, like chess, for example, as was for the longest time seen as the highest, um, level of intellect, right?

Rodney Brooks 30:14 Until we got computers who were better at it than people and then we realiz- You know, but if you go back to the '90s, you'll see, you know, the stories in the press around when, when, um...... Kasparov was beaten by-

Lex Fridman 30:25 Yeah.

Rodney Brooks 30:25 ... Deep Blue. "Oh, this is the end of all sorts of things. Computers are gonna be able to do anything from now on." And we saw exactly the same stories with AlphaZero, the- the Go-playing program.

Lex Fridman 30:36 Yeah. But still, to me, reasoning is a special thing. And perhaps-

Rodney Brooks 30:42 No, wait. Actually, we're- we're really bad at reasoning.

Lex Fridman 30:44 (laughs)

Rodney Brooks 30:44 We just use these analogies based on our hunter-gatherer intuitions, and-

Lex Fridman 30:48 But why is that not... Don't you think the ability to construct metaphor is a really powerful thing?

Rodney Brooks 30:54 Oh, yeah, it is.

Lex Fridman 30:54 Like, to tell stories.

Rodney Brooks 30:55 It is.

Lex Fridman 30:56 Yeah.

Rodney Brooks 30:56 That's, uh, it's the constructing the metaphor and registering that-

Lex Fridman 30:59 Yes, that registration.

Rodney Brooks 30:59 ... to something constant in our brains.

Lex Fridman 31:01 Like, isn't that what we're doing with- with vision, too, and we're- we're telling our s- stories, we're constructing good models of the world?

Rodney Brooks 31:08 Yeah, yeah. But, uh, but, um, I think we- we jumped between what we're capable of and how we're doing it.

Lex Fridman 31:16 Right. Yeah, sure.

Rodney Brooks 31:16 There- there was a little confusion that went on-

Lex Fridman 31:18 Sure. (laughs)

Rodney Brooks 31:18 ... um, uh, as- as we're- we're telling each other stories then.

Lex Fridman 31:21 Yes, exactly. (laughs) Trying to delude each other. No, I- I just think, uh, I'm not exactly, so I'm trying to pull apart this Moravec's paradox. It's-

Rodney Brooks 31:30 I don't view it as a paradox.

Lex Fridman 31:32 (laughs)

Rodney Brooks 31:33 What- what did- what did evolution, what did evolution spend its time on?

Lex Fridman 31:36 Yes.

Rodney Brooks 31:36 It spent its time on getting us to perceive and move in the world. That was, you know, 600 million years as multi-celled creatures doing that. And then it was, you know, relatively recent that we, that we, you know, were able to, um, hunt or- or gather or, you know, even- even animals hunting, that's much more recent. And then, and then, uh, anything that we, you know, speech, uh, language, those things are, you know, just a couple of hundred thousand years probably-

Lex Fridman 32:03 Yeah.

Rodney Brooks 32:04 ... if a, if- if- if that long, and then agriculture, 10,000 years, you know? All that stuff was built on top of those earlier things which took a long time to develop.

Lex Fridman 32:14 So if you then look at the engineering of these things, so building it into robots, what's the hardest part of robotics do you think? As, uh, through the decades that you worked on robots, uh, in the context of what we're talking about, visio- like on perception, the actual sort of the- the biomechanics of movement. Uh, I'm kinda drawing parallels here between humans and machines always. Like, uh, what do you think is the hardest part of robotics?

Rodney Brooks 32:46 I- I sorta think all of them. (laughs)

Lex Fridman 32:49 (laughs)

Rodney Brooks 32:49 There are no easy parts to do well. Um, we- we sort of go reductionist and we reduce it to, oh, if only we had all the- the location of all the points in 3D-

Lex Fridman 32:59 Yeah.

Rodney Brooks 32:59 ... things would be great.

Lex Fridman 33:01 (laughs)

Rodney Brooks 33:01 Mm. You know, if only we had labels on the- on the images, you know, things would be great. But, you know, as- as we see, that's not good enough. Uh, uh, some deeper understanding...

Lex Fridman 33:13 But if you, if I came to you and I could solve one category of problems in robotics, uh, instantly, what would n- give you, uh, the greatest pleasure? (laughs)

Rodney Brooks 33:26 (laughs)

Lex Fridman 33:27 Is, uh, I mean, is it, um, you know, you- you look at robots that manipulate objects, uh, wha- what's hard about that, you know? Is it, uh, the perception? Is it the, um, the reasoning about the world, like common sense reasoning? Is it the actual building a robot that's able to interact with the world? Is it like human aspects of a robot that's interacting with humans and that- that game theory of how they work well together?

Rodney Brooks 33:56 Well, let's talk about manipulation for a second-

Lex Fridman 33:58 Sure.

Rodney Brooks 33:58 ... because I had this really blinding moment. Uh, uh, you know, I'm a grandfather, so grandfathers have blinding moments.

Lex Fridman 34:06 Yes.

Rodney Brooks 34:06 Just, uh, um, three or four miles from here, um, last year, my, uh, 16-month-old grandson was in his new house, first time, right? First time in his house. And he'd never, uh, been able to get to a window before, but this had some low windows.

Lex Fridman 34:24 Mm-hmm.

Rodney Brooks 34:25 And he goes up to this window with a handle on it that he's never seen before and he's got one hand pushing the window and the other hand turning the handle to open the window.

Lex Fridman 34:35 Mm-hmm.

Rodney Brooks 34:36 He- he- he knew, he, two different hands-

Lex Fridman 34:40 (laughs)

Rodney Brooks 34:40 ... two different things he knew how to- how to put together-

Lex Fridman 34:43 Yeah.

Rodney Brooks 34:44 ... and he's 16 months old.

Lex Fridman 34:45 And there you are watching in awe. (laughs)

Rodney Brooks 34:49 (laughs) Yeah.

Lex Fridman 34:50 Well, how did- how did-

Rodney Brooks 34:52 In an environment- environment he'd never seen before, a mechanism he'd never seen.

Lex Fridman 34:55 How did he do that? How did he do that? How did he do that?

Rodney Brooks 34:56 Yes, that's a good question, how did he do that? That's why... (laughs)

Lex Fridman 35:00 It's like, okay, like, uh, you could see- the- the leap of genius from using one hand to perform a task to combining d- doing... I mean, first of all, in manipulation, that's really difficult. It's like two hands both necessary to complete the action.

Rodney Brooks 35:16 And completely different and he'd never seen a window open before.

Lex Fridman 35:20 Yeah. Huh.

Rodney Brooks 35:21 (laughs)

Lex Fridman 35:22 (laughs)

Rodney Brooks 35:22 But he inferred somehow, handle, open something-

Lex Fridman 35:25 Yeah.

Rodney Brooks 35:25 ... a door.

Lex Fridman 35:26 There may have been a lot of, um, slightly different failure cases that you didn't see.

Rodney Brooks 35:32 Yeah.

Lex Fridman 35:32 And not with a window, but with other objects of turning and twisting and handles.

Rodney Brooks 35:36 Oh, well, the- the- you know, there's a gr- there's a great counter to, um, you know, reinfors- reinforcement learning. We'll just give, you know, the- the robot, um, we will give the robot plenty of time to try everything.

Lex Fridman 35:49 Yes.

Rodney Brooks 35:49 Actually, can I tell a little side story here? So I'm in, um, DeepMind in London, um, this was three, four years ago, where, um, you know, there's a big Google building and then you go inside and then you go through this... More security, and then you get to DeepMind where the other Google employees can't go.

Lex Fridman 36:08 Yeah.

Rodney Brooks 36:09 And I'm in a, I'm in a conference room, Bay A conference room with some of the people, and they tell me about their reinforcement learning experiment with, uh, robots, um, um, which, um, are just trying stuff out, and they're my robots.... they're, they're Sawyers that we sold them. Um, uh, and they really like 'em 'cause Sawyers are compliant and can sense forces so they don't break when they're s- bashing into walls-

Lex Fridman 36:35 Mm-hmm.

Rodney Brooks 36:36 ... they, they stop and they do all this stuff. And, you know, so you just let the robot do stuff and eventually it figures stuff out.

Lex Fridman 36:42 By the way, Sawyer, we're talking about robot manipulation, so robot arms and so on.

Rodney Brooks 36:47 Yeah, Sawyer- Sawyer is a robot, um-

Lex Fridman 36:49 Yeah. Just, you know, wh- what's Sawyer?

Rodney Brooks 36:51 Sawyer is a robot arm that my company, Rethink Robotics-

Lex Fridman 36:54 Yeah.

Rodney Brooks 36:54 ... built.

Lex Fridman 36:55 Thank you for the context. (laughs)

Rodney Brooks 36:56 Yeah, sorry.

Lex Fridman 36:57 Okay, cool. So we're in deep mind.

Rodney Brooks 36:59 A- and the, you know, it's in the next room, these robots are just bashing around to try and use reinforcement learning to learn how to act. And, "Well, can I go see them?" "Oh, no, they're secret." They were my robots-

Lex Fridman 37:09 Yeah.

Rodney Brooks 37:09 ... that were secret. (laughs)

Lex Fridman 37:11 That's hilarious. Okay.

Rodney Brooks 37:12 Anyway, the point is, you know, this idea that you just let, uh, reinforcement learning figure everything out is so counter to how a kid does stuff. So, again, story about my grandson, I gave him this, this, uh, uh, box that had lots of different lock mechanisms. He didn't randomly, you know, and he was 18 months old, he didn't randomly try to touch every surface or push everything, he found, he could see what, where the mechanism was and he started exploring the mechanism-

Lex Fridman 37:42 Mm-hmm.

Rodney Brooks 37:42 ... for each of these different lock mechanisms. He, and there was reinforcement, no doubt, of some sort going on there. But he applied a pre-filter which cut down the search space dramatically.

Lex Fridman 37:54 Absolutely. I, I, I wonder to what level we're able to introspect what's going on, because what's also possible is you have something like reinforcement learning going on in the mind in the space of imagination, so, like, you have a good model of the world you're predicting and you may be running those tens of thousands of, like, loops, but you're, like, as a human, you're just looking at yourself trying to tell a story of what happened and it might seem simple but maybe there's a lot of computation going on.

Rodney Brooks 38:24 Whatever it is. But there's also a mechanism that's been built up-

Lex Fridman 38:28 Yeah.

Rodney Brooks 38:28 ... it's not just random search.

Lex Fridman 38:31 Yeah.

Rodney Brooks 38:31 That mechanism-

Lex Fridman 38:32 Yeah.

Rodney Brooks 38:32 ... prunes it dramatically.

Lex Fridman 38:34 Yeah, the, that, that pruning, uh, that pruning step. But it doesn't... it's possible that that's... so you don't think that's akin to a neural network inside a reinforcement learning algorithm? Is it possible?

Rodney Brooks 38:50 It's, uh, yeah, until-

Lex Fridman 38:52 (laughs)

Rodney Brooks 38:52 ... it's possible, I, uh, but, but I, you know, um, I, I s- I'll be incredibly surprised if that happens. I'll also be incredibly surprised that, you know, after all the decades that I've been doing this where every, every few years someone thinks, "Now we've got it."

Lex Fridman 39:09 Mm-hmm.

Rodney Brooks 39:10 "Now we've got it." You know, four or five years ago I was saying, "I don't think we've got it yet." And everyone was saying, "Oh, you don't understand how powerful AI is." I had people tell me, "You don't understand how powerful it is." Um, I, you know, I, I, I sorta had a, a, a track record of what the world had done to think, "Well this is no different from before."

Lex Fridman 39:31 Yeah.

Rodney Brooks 39:31 "Oh, we have bigger computers? We had bigger computers in the '90s and we could do more shit, stuff."

Lex Fridman 39:36 Yeah.

Rodney Brooks 39:36 (laughs)

Lex Fridman 39:38 (laughs) But, okay, so let me, let me, let me push back 'cause I, I'm, I'm generally sort of optimistic and try to find the beauty in things. Um, I think there's a lot of, uh, surprising and beautiful things that neural networks, this new generation of deep learning revolution has revealed. To me, it has continually been very surprising the kind of things it's able to do. Now-

Rodney Brooks 40:01 Oh.

Lex Fridman 40:01 ... generalizing that over saying, like, this, "We've solved intelligence," that's another, uh, big leap, but is there something surprising and beautiful to you about neural networks that, where actually you sat back and said, "I, I did not expect this."

Rodney Brooks 40:18 Oh, I think, I think their performance, their performance on ImageNet was shocking.

Lex Fridman 40:24 So computer vision, those early days was just very, like, wow, okay.

Rodney Brooks 40:28 That doesn't mean that they're solving everything in computer vision we need to solve, or in vision for robots.

Lex Fridman 40:35 What about AlphaZero and self-play mechanisms and reinforcement learning? Isn't that-

Rodney Brooks 40:39 Yeah, that was all in, in Donald Mickey's 1961 paper, (laughs) um, everything there was there, wh- which introduced reinforcement learning. Um-

Lex Fridman 40:48 No, but come on. So, no, you're talking about the actual techniques, but isn't it surprising to you the level it's able to achieve with no human supervision of chess play? Like, I, to me, there's a big, big difference-

Rodney Brooks 41:04 Maybe, mayb-

Lex Fridman 41:04 ... in Deep Blue and-

Rodney Brooks 41:06 ... maybe what that's saying is how overblown our view of ourselves is. You know, we-

Lex Fridman 41:13 That chess is easy? (laughs)

Rodney Brooks 41:15 Yeah, is tha-

Lex Fridman 41:16 (laughs)

Rodney Brooks 41:16 Yeah, I mean, I, I, uh, I came across this 1946 report-

Lex Fridman 41:25 Mm-hmm.

Rodney Brooks 41:25 ... that, um... and I'd seen this as a kid in one of those books that my mother had given me actually. Um, 1946 report which pitted, uh, someone with an abacus against an electronic calculator-

Lex Fridman 41:39 Mm-hmm.

Rodney Brooks 41:40 ... and he beat the electronic calculator. You know, so there, at that point was, well, humans are still better than-

Lex Fridman 41:47 Right.

Rodney Brooks 41:47 ... machines at calculating. Are you surprised today that a machine can, you know, do a billion floating point operations a second and, you know, you're, you're puzzling for, for minutes to do one? So, you know-

Lex Fridman 42:01 I, I am s- I mean, that, I don't know, but I am certainly surprised... there's something, uh, to me different about learning. So a system that's able to learn.

Rodney Brooks 42:12 Learning. Now, see, see now you're getting to one of the, uh, deadly sins.

Lex Fridman 42:15 Mm-hmm.

Rodney Brooks 42:17 Um...

Lex Fridman 42:17 Be- because (laughs) of using terms, uh, overly broadly? (laughs)

Rodney Brooks 42:20 Yeah, I mean, there's so many different forms of learning.

Lex Fridman 42:23 Yeah.

Rodney Brooks 42:23 And so many different forms. You know, I learnt my way around the city, I learnt to play chess, I learnt Latin, um, I learnt to ride a bicycle. All of those are, you know, are very different capabilities-

Lex Fridman 42:35 Yeah.

Rodney Brooks 42:35 ... and if someone, you know, has a-... well, in, uh, you know, in the old days people would write a paper about learning something. Now, the corporate, uh, press office puts out a press release about how company X has, has, is leading the world, because they have a system that can-

Lex Fridman 42:57 Yeah, but h- here's the thing. Okay, so what is learning? What I refer to-

Rodney Brooks 43:01 Learning is many things.

Lex Fridman 43:02 But I, uh-

Rodney Brooks 43:03 It's a suitcase word.

Lex Fridman 43:04 It's a suitcase word.

Rodney Brooks 43:05 It's a suitcase word.

Lex Fridman 43:05 But lo- loosely, there's a dumb system, and over time it becomes smart.

Rodney Brooks 43:13 Well, it becomes less dumb at the thing that it's doing.

Lex Fridman 43:16 Yeah.

Rodney Brooks 43:16 Smart is a d-

Lex Fridman 43:17 Yes, that's true.

Rodney Brooks 43:17 Smart is a loaded word.

Lex Fridman 43:19 Yes. Less, less dumb at the thing it's doing-

Rodney Brooks 43:21 Again, gets better performance under some measure-

Lex Fridman 43:23 Yeah.

Rodney Brooks 43:24 ... under some set of conditions, at that thing. And, and most of these learning algorithms, um, uh, learning systems, fail when you change the conditions just a little bit, in a way that humans don't. So-

Lex Fridman 43:38 Right.

Rodney Brooks 43:39 ... I was at DeepMind, um, the AlphaGo had just come out.

Lex Fridman 43:43 Mm-hmm.

Rodney Brooks 43:44 And I said, "What would've happened if you'd given it a 21 by 21 board instead of a 19 by 19 board?" They said, "Failed totally." But a human player would actually, you know, well, would actually be able to play that game.

Lex Fridman 43:55 And actually funny enough, if you look at DeepMind's work, uh, since then, uh, they are pres- uh, they're presenting a lot of algorithms that would do, uh, the, that would do well on the, at the, at the bigger board. So they are slowly expanding this generalization. I mean, to me there is a core element there. I (laughs) it is very surprising to me that even in a constrained game of chess or Go, that through self-play, by a system playing itself, that can en- it can achieve s- superhuman level performance through learning alone. So like-

Rodney Brooks 44:30 Okay. So, so, you know, you, you didn't, you, you didn't-

Lex Fridman 44:32 ... it's, it's so fundamentally different in search of that

Rodney Brooks 44:34 ... you didn't like it when I referred to Donald Mickey's 1961 paper?

Lex Fridman 44:40 Yeah.

Rodney Brooks 44:40 There, in the second part of it-

Lex Fridman 44:42 Yes.

Rodney Brooks 44:43 ... which came a year later, they had self-play on an electronic computer-

Lex Fridman 44:47 Yes.

Rodney Brooks 44:47 ... at tic-tac-toe, okay.

Lex Fridman 44:49 Yeah.

Rodney Brooks 44:49 It's not as ... but it learnt to play tic-tac-toe through self-play.

Lex Fridman 44:52 No, no, that, that's not ... and it-

Rodney Brooks 44:53 And it learned to play optimally.

Lex Fridman 44:54 What I'm saying is, uh, I, okay, I have a little bit of a bias, but I, I, I find ideas beautiful, but only when they actually realize the promise. That's another level of beauty. Like, for example, uh, what, uh, uh, Bezos and Elon Musk are doing with rockets, we had rockets for a long time, but doing reusable cheap rockets, it's very impressive. In the same way, I ... okay.

Rodney Brooks 45:21 Yeah.

Lex Fridman 45:21 I would've not predicted ... first of all, when I was, uh, s- s- started and fell in love with AI, the game of Go was seen to be impossible to solve. Okay. So I thought maybe, you know, I ... maybe it'd be possible to maybe have big leaps in a Moore's Law style of way in computation that would be able to solve it. But I would never have guessed that you could learn your way however ... uh, I mean, in the narrow sense of learning, learn your way to b- to, to beat the best people in the world at the game of Go without human supervision, not studying the game of experts. But-

Rodney Brooks 46:01 Okay, so, so use-

Lex Fridman 46:02 ... that's surprising.

Rodney Brooks 46:03 ... using, using a different learning technique-

Lex Fridman 46:04 Yes.

Rodney Brooks 46:05 ... Arthur Samuel, in the early '60s, and he was the first person to use machine learning-

Lex Fridman 46:11 Yeah.

Rodney Brooks 46:11 ... got ri- had a program that could beat the world champion at checkers. Now-

Lex Fridman 46:16 Yes.

Rodney Brooks 46:17 So, and that at the time was considered amazing.

Lex Fridman 46:20 Yes.

Rodney Brooks 46:20 By the way, Arthur Samuel had some fantastic advantages.

Lex Fridman 46:23 Yeah.

Rodney Brooks 46:23 Do you wanna hear Arthur Samuel's-

Lex Fridman 46:24 Yeah, yeah.

Rodney Brooks 46:24 ... advantages? Two things. One, he was at, uh, the 1956, um, AI conference. I knew Arthur later in life.

Lex Fridman 46:32 Mm-hmm.

Rodney Brooks 46:32 Uh, he was at Stanford when I was a grad student there. He wore a tie and a jacket every day-

Lex Fridman 46:36 Right.

Rodney Brooks 46:36 ... when the rest of us didn't. Um, he's delightful man, delightful man. Um, uh, it turns out Claude Shannon, in a 1950 Scientific American article, uh, outlined the w- uh, on chess playing, outlined the learning mechanism that Arthur Samuel used, um, and they had met in 1956. I assume there was some communication, but I don't know that for sure. But Arthur Samuel had been a, a vacuum tube engineer, um, getting reliability of vacuum tubes, and then had overseen, um, the first transistorized computers at IBM. And in those days, before you shipped a computer, you ran it for a week to see, to get early failures. So he had this whole farm of computers running random code-

Lex Fridman 47:23 Mm-hmm.

Rodney Brooks 47:23 ... um, for hours and hours, um, uh, a week, uh, for each computer. He had a whole bunch of them. So he ran his chess learning program, uh, with self-play-

Lex Fridman 47:34 Mm-hmm.

Rodney Brooks 47:35 ... uh, on the, on IBM's production line. He had more com- computation available to him than anyone else in the world, and then he was able to produce a chess playing program, I mean, uh, a checkers playing program that could beat the world champion.

Lex Fridman 47:49 Mm-hmm.

Rodney Brooks 47:50 So-

Lex Fridman 47:50 That's amazing. The, the question is, when I mean surprised, I don't just mean it's nice to have that accomplishment, is there is a stepping towards something that feels, uh, more intelligent than before.

Rodney Brooks 48:06 Yeah, but that's-

Lex Fridman 48:06 And the question is-

Rodney Brooks 48:07 ... that's, that's in your view of the world, but-

Lex Fridman 48:08 Okay, okay. Well, let me then ... um, it doesn't mean I'm wrong.

Rodney Brooks 48:12 No.

Lex Fridman 48:12 (laughs)

Rodney Brooks 48:13 No, it doesn't. (laughs)

Lex Fridman 48:14 (laughs) So the question is, if we keep taking steps like that, how far that takes us. Are we going to build better recommender systems? Are we going to build slightly-

Rodney Brooks 48:23 Well, okay, so-

Lex Fridman 48:23 ... better robots? Or will we solve intelligence?

Rodney Brooks 48:26 So, you know, I'm putting my bet on, um, we're still missing a, a whole lot.

Lex Fridman 48:33 A lot.

Rodney Brooks 48:34 Um, and, and why would I say that? Well, in these games they're all, um, you know, 100% information games.... but again, but e- each of these systems is a very, uh, short description of the current state, um, which is different from registering and perception in the world.

Lex Fridman 48:53 Okay.

Rodney Brooks 48:53 Which gets back to Moravec's paradox.

Lex Fridman 48:55 I'm definitely not saying that, uh, uh, chess is somehow harder than, uh, perception or uh, e- any kind of, e- even, any kind of robotics in the physical world I- I definitely think is- is way harder than the game of chess. So, I'm- I was always much more impressed by like c- the workings of the human mind is incredible. The human mind's incredible. I've, uh, I- I believe that from the very beginning. I wanted to be a psychiatrist for the longest time. I always thought that's way more incredible than the game of chess. I think the game of chess is, uh, is... I love the Olympics. It's- it's just another example of us humans picking a task and then agreeing that a million humans, will dedicate their whole life to that task, and that's the cool thing that the human mind is able to focus on one task and then compete against each other and achieve like weirdly incredible levels of performance. That's the aspect of chess that's super cool, not that chess in itself, uh, is really difficult. It- it's like the Fermat's Last Theorem is not in itself to me that interesting. The fact that, uh, thousands of people have been struggling to solve that particular problem is- is fascinating.

Rodney Brooks 49:58 So, can I tell you my disease in this way?

Lex Fridman 50:00 Sure. (laughs)

Rodney Brooks 50:01 Uh, whi- which actually is closer to what you're saying.

Lex Fridman 50:03 Yeah.

Rodney Brooks 50:03 So as a child, you know, I was building various, I called them computers, they weren't general purpose computers.

Lex Fridman 50:09 Ice cube tray.

Rodney Brooks 50:10 The ice cube tray was one, but I built other machines and what I liked to build was machines that could beat adults at a game.

Lex Fridman 50:16 Yeah.

Rodney Brooks 50:17 And they couldn't, the adults couldn't beat my machine.

Lex Fridman 50:19 Yes.

Rodney Brooks 50:20 So that was... (laughs)

Lex Fridman 50:20 So you were like, uh... that's powerful. Like that's a- that's a way to rebel.

Rodney Brooks 50:25 Yeah.

Lex Fridman 50:26 I- I, by the way, um, did you... when was the first time you built something that outperformed you? Do you remember? Like...

Rodney Brooks 50:34 Well, a- a- and I knew how it worked, I was probably nine years old and I built a thing that, uh, it was a game where you, you take turns in taking matches from a pile and, uh, either the one who takes the last one or the one who doesn't take the last one wins, I forget. And so it was pretty easy to build that out of wires and nails and little coils that were like plugging in the number and, uh, a few light bulbs. Um, the one, the one I was prouder of, I was 12 when I- I built a- a- a thing out of old, um, telephone switchboard switches that could, uh, always, uh, win at tic-tac-toe.

Lex Fridman 51:11 Mm-hmm.

Rodney Brooks 51:11 And that was a much harder circuit-

Lex Fridman 51:13 (laughs)

Rodney Brooks 51:14 ... to design.

Lex Fridman 51:14 Yeah.

Rodney Brooks 51:14 But again, was just... it- it was no active components, it was just three position switches, empty, X, zero, or, uh, O, and, um, and nine of them and- and a light bulb on which- which move it wanted next and then-

Lex Fridman 51:29 Yeah.

Rodney Brooks 51:30 ... the human would go and move that for them.

Lex Fridman 51:31 See there's magic in that creation. I- I-

Rodney Brooks 51:33 It was, yeah. Yeah.

Lex Fridman 51:34 I tend to, uh, I tend to see magic in robots that... like, I- I also think that intelligence is, uh, is a little bit overrated. I think we can have deep connections with robots very soon. And...

Rodney Brooks 51:49 Oh, we'll come- we'll come back to connections with robots.

Lex Fridman 51:52 Sure.

Rodney Brooks 51:52 But- but I do want to say, I- I don't... I think peop- too many people make the mistake of seeing that magic and thinking, "Oh, it will just continue." You know? But each- each one of those is a hard-fought battle for the next step, the next step.

Lex Fridman 52:07 Yes. I mean, the open question here is, and this is why I'm playing devil's advocate, but I often do when I read (laughs) your blog post in my mind because I have like this eternal optimism, is it's not clear to me... so I don't do what obviously the journalists do or like give into the hype, but it's not obvious to me how many steps away we are from... from a truly transformational understanding of what it means, uh, to build intelligent systems like or how to build intelligent systems. I'm also aware of the whole history of artificial intelligence which is where your deep grounding of this is, is there's been an optimism for decades. (laughs) And that optimism, just like reading old optimism is absurd because people were like, "This is..." they were saying things are trivial for decades since the '60s. They were saying everything is tru- computer vision is trivial. But I think my mind is working crisply enough to where... I- I mean we can dig into it if- if you want, I'm- I'm really surprised by w- the things DeepMind has done, I don't think they're s- they're yet, um, close to solving intelligence but I'm not sure it's not 10- uh, 10 years away. What I'm referring to is interesting to see when the engineering, um, it takes that idea to scale and this... and the idea works.

Rodney Brooks 53:33 And- and no, it fools people.

Lex Fridman 53:35 Okay. Honestly, Rodney, if it was you, me, and Demis inside a room, forget the press, forget all those things, just as a scientist, as a roboticist, you don't... that wasn't surprising to you that at scale, so we're talking about very large now, we'll get... okay, let's pick one that's the most surprising to you. Okay, please don't yell at me. GPT-3 (laughs) . Okay. Well hold on a second, hold on a second.

Rodney Brooks 53:57 I was gonna- I was gonna bring that up.

Lex Fridman 53:59 Okay. Thank you. (laughs) AlphaZero, AlphaGo, AlphaGo Zero, AlphaZero and then AlphaFold 1 and 2. So are... are any of... do- do any of these kind of have this core of, uh... n- now forget usefulness or application and so on which you could argue for AlphaFold, like as a scientist, was those surprising to you that it worked as well as it did?

Rodney Brooks 54:24 Uh, okay. So if we're gonna make the distinction between surprise and usefulness, and- and- and- I'll- I'll... I have to explain this.

Lex Fridman 54:33 Sure.

Rodney Brooks 54:34 I would say AlphaFold. And one of the problems at the moment with AlphaFold is...... you know, it gets a lot of them right. Which is a surprise to me, 'cause they're a really complex thing. Uh, but you don't know which ones it gets right.

Lex Fridman 54:49 Mm-hmm.

Rodney Brooks 54:49 Um, which then is a- is a bit of a problem. Now they've come out with a recent-

Lex Fridman 54:53 You mean the structure of the proteins, it gets a lot of those right?

Rodney Brooks 54:56 Yeah.

Lex Fridman 54:56 Yeah.

Rodney Brooks 54:56 Gets a, a surprising number of them right.

Lex Fridman 54:58 Yeah.

Rodney Brooks 54:58 Uh, it's been a really hard problem. So that was a surprise how many it gets right. So far the usefulness is limited because you don't know which ones are right or not, and th- now they've come out with a, um, a thing in the last few weeks which is trying to get a useful tool out of it, and they may well do it. Um-

Lex Fridman 55:15 In that sense at least AlphaFold is different, because you're, uh, AlphaFold2 is different, because now i- it's producing datasets that are actually, uh, you know, potentially revolutionizing computational biology. Like they will actually help a lot of people. But-

Rodney Brooks 55:31 Y- you would say prevent- uh, potentially revolutionizing.

Lex Fridman 55:34 Yeah.

Rodney Brooks 55:34 We don't know yet. But, yeah.

Lex Fridman 55:36 That's true, yeah. But there, you know, but I- I- I got you. I mean this is, okay, so you know what? This is gonna be so fun. So let's go right

Lex Fridman 55:45 into it. Speaking of robots that operate in the real world, let's talk about self-driving cars.

Rodney Brooks 55:53 Oh. (laughs)

Lex Fridman 55:54 (laughs) Okay. Because you do, you have built robotics companies. You're- you're one of the greatest roboticists in history and that's not in space of i- just in the space of ideas, we'll also probably talk about that, but in the actual building and execution of businesses that make robots that are useful for people and that actually work in the real world and make money. You also sometimes are critical of Mr. Elon Musk, or let's more specifically focus on this particular technology which is autopilot inside Teslas. What are your thoughts about Tesla autopilot or more generally vision-based machine learning approach to semi-autonomous driving? Uh, th- these are robots that are being used in the real world by hundreds of thousands of people and, uh, w- if you want to go there I can go there but let's not too much, which they're, let's say they're on par safety-wise as humans currently, meaning human alone versus human plus robot.

Rodney Brooks 56:58 Okay. So first let me say I really like the car I came here- in here today.

Lex Fridman 57:04 Which is?

Rodney Brooks 57:05 Um, a 2021, um, model, uh, Mercedes E450.

Lex Fridman 57:12 Mm-hmm.

Rodney Brooks 57:13 I am impressed by the, um, m- machine vision. So are now other things. I'm impressed by what it can do. I'm really impressed, um, with, uh, many aspects of it. And- and I'm, um ...

Lex Fridman 57:29 It's able to stay in lane? Is it, uh-

Rodney Brooks 57:31 Oh yeah, it does the- it does the lane stuff. Um, it, uh, uh, you know, it- it's- it's looking at- uh, I mean the side of me, it's telling me about nearby cars.

Lex Fridman 57:40 For blind spots and so on?

Rodney Brooks 57:41 Yeah. When I- when I- when I- when I'm going in close to something to park, I get this beautiful, gorgeous, top-down view of the world. I am impressed up the wazoo of how, you know, registered and m- metrical that is.

Lex Fridman 57:56 Oh, so it's like multiple cameras and it's all very connected together-

Rodney Brooks 57:59 Yeah, uh, uh-

Lex Fridman 57:59 ... to produce the 360 view kind of thing?

Rodney Brooks 58:01 360 view, uh, you know, it's synthesized ...

Lex Fridman 58:03 Okay.

Rodney Brooks 58:03 ... so it's above the car. And it is unbelievable. I- I got this car in January. It's the longest I've ever owned a car without digging it. Um, so it's better than me. It- it-

Lex Fridman 58:13 Wow, okay.

Rodney Brooks 58:13 ... or me and it together, uh, are better. So I'm not saying technology's, um, uh, bad or not useful, but here's my point.

Lex Fridman 58:25 Yes.

Rodney Brooks 58:27 It's just, it's a replay of the same movie.

Lex Fridman 58:30 Mm-hmm.

Rodney Brooks 58:31 Okay? So maybe you've seen me ask this question before, but, um, when, um, when- when did the first car go over 55 miles an hour for over, um, over 10 miles on a public freeway with other traffic around driving completely autonomously? When did that happen?

Lex Fridman 58:59 The- is it- was it CMU in the '80s or something? I- it was a long time ago.

Rodney Brooks 59:02 It was actually 1987 in- in Munich. Um ...

Lex Fridman 59:05 Oh, Munich, yeah, yeah.

Rodney Brooks 59:06 Uh, at the Bundeswehr.

Lex Fridman 59:07 Yes.

Rodney Brooks 59:08 Um, um, um ... So they- they had it running in 1987. When do you think, and Elon has said he's gonna do this, when do you think we'll have the first car drive coast to coast in the US hands off the wheel, hands off the wheel, feet off the pedals coast to coast?

Lex Fridman 59:26 As far as I know a few people have claimed to do it.

Rodney Brooks 59:28 1995. That was Carnegie Mellon.

Lex Fridman 59:30 I didn't know, oh, that was the co-

Rodney Brooks 59:32 Yeah.

Lex Fridman 59:33 Uh, they didn't claim ... Uh, did they claim 100%? They claimed-

Rodney Brooks 59:35 Not- not 100%.

Lex Fridman 59:36 Yeah, yeah.

Rodney Brooks 59:37 Not 100%. But- but my-

Lex Fridman 59:38 And then there's a few marketing people who have claimed 100% since then.

Rodney Brooks 59:42 Yeah. But my point is that, you know, I- I- I, what I see happening again is someone sees a demo and they over-generalize and say, "We must be almost there."

Lex Fridman 59:52 Yes.

Rodney Brooks 59:52 Well, we've been- we've been working on it for 35 years.

Lex Fridman 59:55 Okay. But that's demos.

Rodney Brooks 59:56 (laughs)

Lex Fridman 59:56 But this is gonna take us back to the same conversation with AlphaZero. Are you not ... Okay, I'll just say what I am, because I thought ... Okay, when I first, uh, started interacting with, uh, with the Mobileye implementation of Tesla autopilot, it's- I've driven a lot of car- you know, I- I've been in Google self-driving car since the beginning. Um, I thought there was no way, before I sat and used Mobileye, I thought there- just knowing computer vision, I thought there's no way it could work as well as it was working. So my model of the limits of computer vision, uh, was- was way more limited than the actual implementation of Mobileye. I was- so that's one example. I was really surprised. I was like, "Wow, that was- that was incredible." The second surprise came when Tesla threw away Mobileye ...... and started from scratch. Uh, I thought there's no way they can catch up to Mobileye. I thought what Mobileye was doing was kind of incredible, like the amount of work and the annotation.

Rodney Brooks 1:00:57 Yeah. Well, Mobileye w- started with Amnon Shashua and, and used a lot of traditional, you know, hard-fought computer vision techniques.

Lex Fridman 1:01:05 But they also did a lot of good sort of, uh, like non-research stuff, like actual, like-

Rodney Brooks 1:01:11 Oh.

Lex Fridman 1:01:12 ... uh, just good, like what-

Rodney Brooks 1:01:13 Yeah. Yeah.

Lex Fridman 1:01:13 ... you do to make a successful product, right?

Rodney Brooks 1:01:15 Yeah.

Lex Fridman 1:01:15 The scale, the... all that kind of stuff.

Rodney Brooks 1:01:16 Yeah.

Lex Fridman 1:01:17 And so I was very surprised when they, from scratch, were able to catch up to that. Uh, that's very impressive. And I've talked to a lot of engineers that was involved. This is... that was impressive. Uh, and the recent progress, especially under, um, well, with the involvement of Andrej Karpathy, the... what they were... what they're doing with, uh, the data engine, which is converting into... the driving task into these multiple t- tasks and then doing this edge case discovery when they're pulling back. Like, the level of engineering made me rethink what's possible. I don't... I still, you know, um... I don't know to that intensity, but I always thought it was very difficult to solve autonomous driving with all the sensors, with all of the computation. I just thought it was a very difficult problem. But I've, I've been continuously surprised how much you can engineer, first of all, the data acquisition problem 'cause I thought, you know, just because I worked with a lot of car companies an- and, uh, they're, they're so a little, a little bit old school to where I didn't think they could do this at scale, like AWS style data collection. So when Tesla was able to do that, uh, I started to think, okay, so what are the limits of this? Now, I still believe that, um, a driver, like sensing and the interaction with the driver and, like studying the human factors psychology problem is essential. It's- it's always going to be there. Uh, it's always going to be there, even with fully autonomous driving. But I've been surprised what is the limit, especially a vision-based alone, how far that can take us. Um, so that's my levels of surprise. Now... (laughs)

Rodney Brooks 1:03:02 Okay.

Lex Fridman 1:03:03 Uh, can, can you explain, uh, in the same way you said like AlphaZero, that's a homework problem that's scaled large and it's chess, like who cares? Go with... Here's actual people using an actual car and driving... many of them drive more than half their miles using this system.

Rodney Brooks 1:03:21 Right. So an A... yeah. He... they're doing well with- with pure vision. Uh-

Lex Fridman 1:03:26 With pure vision, yeah.

Rodney Brooks 1:03:27 ... and

Lex Fridman 1:03:27 you know, they... (laughs) And now no radar, which is...

Rodney Brooks 1:03:30 I suspect that can't go all the way, and one reason is w- without- without new cameras that have a dynamic range closer to the human eye, 'cause the human eye has incredible dynamic range and we make use of that dynamic range in- in its, uh... 11 orders of magnitude or some crazy number like that. The cameras don't have that, which is why you see the- the- the- the bad cases where there's sun on a white thing and it blinds i- i- in a way it wouldn't blind a person. I think there's a bunch of things to think about before you say, "This is so good, it's just gonna work."

Lex Fridman 1:04:08 (laughs)

Rodney Brooks 1:04:09 (laughs) Okay? Um, um, and I'll come at it from multiple angles and I know you've got a lot of time.

Lex Fridman 1:04:15 Yeah. (laughs) Okay, let's- let's- let's do this.

Rodney Brooks 1:04:17 But I, but I have thought about these things. Uh-

Lex Fridman 1:04:19 Yeah. I, I know. You've been writing a lot of great blog posts about it for a while before Tesla (laughs) had autopilot, right? So you've been thinking about autonomous driving for a while from every angle.

Rodney Brooks 1:04:31 Yeah. So, so a few things. You know, in the US, um, I think the- the- the death rate from, uh, motor vehicle accidents, uh, is about 35,000 a year, um, which is an outrageous number. Not outrageous compared to COVID deaths, but, you know, there is no rationality, um, and that's part of the thing. People have said... engineers say to me, "Well, if we cut down the number of deaths by 10% by having autonomous driving, that's gonna be great. Everyone will love it." And my prediction is that if autonomous vehicles kill more than 10 people a year, there'll be screaming and hollering, even though 35,000 people a year are being killed by human drivers. It's not rational. It's a different set of expectations.

Lex Fridman 1:05:20 Mm-hmm.

Rodney Brooks 1:05:20 And that will probably continue. Um, so there's that aspect of it. The other aspect of it is that, um, when we introduce new technology, we often change the rules of the game. So when we introduced cars first, uh, you know, into our daily lives, we completely rebuilt our cities and we changed all the laws.

Lex Fridman 1:05:47 Yeah.

Rodney Brooks 1:05:48 Jaywalking was not an offense. That was pushed by the car companies so that people would stay off the roads so there wouldn't be deaths from-

Lex Fridman 1:05:56 Yeah.

Rodney Brooks 1:05:56 ... pedestrians getting hit. We completely changed the structure of our cities and had these foul-smelling things, you know, everywhere around us, um, and you know, now you see pushback in cities like Barcelona is really trying to exclude cars, uh, et cetera. Um, so I- I think that to get to self-driving, we will, um... large adoption. It's not gonna be just take the current situation, take out the driver and put the same car doing the same stuff because the end case is too many. Um, y- y- here's an interesting question. How many, um, fully autonomous, um, uh, train systems do we have in the US?

Lex Fridman 1:06:46 I mean, do you count them as fully autonomous? I don't know, 'cause there usually is a driver, but they're kind of autonomous, right?

Rodney Brooks 1:06:52 No, well, let- let- let's get rid of the driver.

Lex Fridman 1:06:56 ... okay, I don't know. I'll bet-

Rodney Brooks 1:06:58 Fif- it's either 15 or 16. Most, most of them are in airports.

Lex Fridman 1:07:01 Okay.

Rodney Brooks 1:07:02 Um, there's a few that go about five, two that go about five kilometers out of airports.

Lex Fridman 1:07:07 Yeah.

Rodney Brooks 1:07:08 Um, uh, when do you, when, when is the first fully autonomous train system for mass transit expected to operate? Fully autonomously, with no driver-

Lex Fridman 1:07:21 Uh, I don't-

Rodney Brooks 1:07:22 ... in, in a US city?

Lex Fridman 1:07:23 I'm s-

Rodney Brooks 1:07:23 It's expected to operate in 2017 in Honolulu (laughs) .

Lex Fridman 1:07:28 Oh, wow.

Rodney Brooks 1:07:29 It's delayed, but they will get there. BART, by the way, was originally gonna be autonomous, uh, here in the Bay Area.

Lex Fridman 1:07:35 I mean, they're all very close to fully autonomous, right? But, yeah.

Rodney Brooks 1:07:39 Yeah, but, but the, the f- getting the close is the thing and I have, I have, I've often gone on a fully autonomous train in Japan, um, one that goes, uh, out to that fake island in the middle of Tokyo Bay, I forget the name of the... Uh, uh, and, and what do you, what do you see when you look at that? What do you see when you go to a fully autonomous train in a, in a, in a, uh, um, an airport? It's not like regular trains. There's, uh, at every station, there's a double set of doors, um, so that there's a door of the train and there's a door off the, the, um, off the, the platform.

Lex Fridman 1:08:17 Yeah.

Rodney Brooks 1:08:17 Um, and, and this is really visible in this Japanese one because it goes out in, in amongst buildings. The whole track is built so that people can't climb onto it.

Lex Fridman 1:08:28 Yeah.

Rodney Brooks 1:08:29 So, there's engineering that then makes the system safe and makes them acceptable. I think we'll see similar sorts of things happen in the US. What surprised me, I thought, wrongly, um, that we would have special purpose lanes on 101, uh, h- in the Bay Area, the, the leftmost lane, so that it would be normal for Teslas or other cars to move into that lane and then say, "Okay, now it's autonomous."

Lex Fridman 1:09:01 Mm-hmm.

Rodney Brooks 1:09:01 And have that dedicated lane. I was expecting movement to that, you know, five years ago, I was expecting we'd have a lot more of movement towards that. We haven't.

Lex Fridman 1:09:09 Mm-hmm.

Rodney Brooks 1:09:10 And it may be because Tesla's been over-promising by saying there's, you know, calling their system fully self-driving, um, I think they may have been gotten there quicker by collaborating to change the infrastructure.

Lex Fridman 1:09:25 Yes.

Rodney Brooks 1:09:26 You know, this is one of the problems with long haul tracking, um, being autonomous. I think it makes sense on freeways, um, at night for the trucks to go autonomously, um, but then there's the how do you get onto and off of the freeway? What sort of infrastructure do you need for that? Um, do you need to have the human in there to do that or can you get rid of the human? So, I think there's ways to get there, but it's an infrastructure argument, because the long tail of cases is very long and the acceptance of it will not be at the same level as human drivers.

Lex Fridman 1:10:06 So, I, I'm with you still and I w- I was with you for a long time, but I am surprised how, how well, how many edge cases machine learning and vision-based methods can cover. This is what, this is what I'm trying to get, get at is, um... I think there's something fundamentally different with, uh, vision-based methods, uh, and Tesla autopilot, and any company that's trying to do the same.

Rodney Brooks 1:10:32 Okay, well, I'm, I'm not, I'm not gonna argue with you 'cause, you know, I, I... We- we're speculating.

Lex Fridman 1:10:39 Yes. But, uh-

Rodney Brooks 1:10:39 And I, um, you know, and my gut feeling tells me it, uh, i- i- it's gonna be, um... Things will, things will speed up when there is engineering of the environment.

Lex Fridman 1:10:50 Oh, yeah, yeah.

Rodney Brooks 1:10:50 Because that's what happened with every other technology.

Lex Fridman 1:10:52 I, I'm, uh, I'm a bit, I don't know about you, but I'm a bit cynical that, uh, infrastructure which relies on, on government to help out in these cases, um, is j- if you just look at infrastructure in all domains, it just, uh, government always drags behind on the infrastructure. There's, like, there's so many just-

Rodney Brooks 1:11:13 Well, in this country.

Lex Fridman 1:11:15 In this... Sure, sorry, yes. In th- in this country and, and, of course, there's many, many countries that are actually much worse on infrastructure, so this does not-

Rodney Brooks 1:11:22 Oh, yes, many of them much worse and then some that, you know, like high-speed rail, the, like other countries have done much better.

Lex Fridman 1:11:29 I guess, uh, my question is like, uh, which is at, at the core what I was trying to think through here and ask you is, like, how hard is the driving problem s- as it currently stands? So, y- you mentioned, like, we don't want to just take the human out and duplicate whatever the human was doing, but if we were to try to do that, what, how hard is that problem? Because I used to think it's w- way harder, like I, I used to think it's, uh, with vision alone, it w- i- it, it would be three decades, four decades.

Rodney Brooks 1:12:02 Okay, so I, I don't know the answer to this thing I'm about to pose, but I do notice that on Highway 280 here in the Bay Area-

Lex Fridman 1:12:12 Mm-hmm.

Rodney Brooks 1:12:12 ... which largely has concrete surface rather than blacktop surface, the white lines that are painted there now have black boundaries around them.

Lex Fridman 1:12:22 Mm-hmm.

Rodney Brooks 1:12:23 And my l- lane drift system in my car would not work without those black boundaries.

Lex Fridman 1:12:30 Interesting.

Rodney Brooks 1:12:31 So, I don't know whether they've started doing it to help the lane drift, whether it is an i- instance of infrastructure following the technology, but it, but it... My car would not perform as well without that change in the way they paint the line.

Lex Fridman 1:12:45 Unfortunately, really good lane keeping is not as valuable, like, it's orders of magnitude more valuable to have a fully autonomous system. Like-

Rodney Brooks 1:12:55 Yeah. But, but for me (laughs) , lane keeping is really helpful 'cause I'm lousy at it.

Lex Fridman 1:13:01 But you, but you wouldn't pay 10 times, like, um...The problem is there's not financial, like it doesn't make sense to, to, to, uh, revamp the infrastructure to make lane keeping easier. It does make sense to revamp the infrastructure-

Rodney Brooks 1:13:16 Oh, I see.

Lex Fridman 1:13:17 ...if you have a large fleet of autonomous vehicles, now you change what it means to own cars, you change the nature of transportation. I mean, but that, that, for that you need, uh, autonomous whi- uh, vehicles. Let me ask you about Waymo then. Uh, I've gotten a, a bunch of chances to, to ride in, in a Waymo, um, self-driving car, and they're, I don't know if you'd call them self-driving, but-

Rodney Brooks 1:13:41 Well-

Lex Fridman 1:13:41 ...you know.

Rodney Brooks 1:13:41 ...I mean, I, I rode in one before they were called Waymo-

Lex Fridman 1:13:44 Yeah.

Rodney Brooks 1:13:44 ...when I was still at, at X.

Lex Fridman 1:13:45 So there's currently, which was a big leap, another surprising leap I didn't think would happen which is they have no driver currently.

Rodney Brooks 1:13:53 Yeah, in, in Chandler.

Lex Fridman 1:13:55 In Chandler, Arizona, and I think they're thinking of doing that in Austin as well. But they're, they're expanding.

Rodney Brooks 1:14:00 Yeah.

Lex Fridman 1:14:00 But even just that-

Rodney Brooks 1:14:01 Although, although, you know, and I, I, I do an annual, um, checkup on this. So as of late last year, they were aiming for hundreds of rides a week.

Lex Fridman 1:14:12 Yeah.

Rodney Brooks 1:14:12 Not thousands. And, um, there is s- no one in a car, but there's certainly, uh, um, safety, uh, people in the loop and it's not clear how many, you know, what the ratio of cars to safety people is.

Lex Fridman 1:14:27 I, it wasn't, o- obviously they're not 100% transparent about this, but-

Rodney Brooks 1:14:31 No, none of them are 100% transparent.

Lex Fridman 1:14:33 ...well this is the problem.

Rodney Brooks 1:14:33 They're very untransparent.

Lex Fridman 1:14:34 But I, uh, at least the way they're... I, I don't wanna make definitely, but they're saying there's no tele-operation.

Rodney Brooks 1:14:41 Um, y- y- y-

Lex Fridman 1:14:42 So like, they're, I mean, (laughs) okay-

Rodney Brooks 1:14:45 Yeah. And, and, and, and, and that sort of fits with, with, um, YouTube videos I've seen of people being trapped in the car-

Lex Fridman 1:14:52 Yeah.

Rodney Brooks 1:14:52 ...um, by a red cone on the, on the street and they do, they do have rescue vehicles that come.

Lex Fridman 1:14:59 Yeah.

Rodney Brooks 1:14:59 And the, then a person gets in and drives it.

Lex Fridman 1:15:02 Yeah. But isn't it incredible to you, it was to me, to get in a car with no driver and watch the steering wheel turn? Like for somebody who has been studying at least the, certainly the human side of autonomous vehicles for many years, and you've been doing it for way longer, like it was incredible to me that this was actually, could happen. I don't care if that scale is 100 cars, this is not a demo. This is not, this is me as a, as a, a regular human-

Rodney Brooks 1:15:28 Yeah. No, the, the argument I have-

Lex Fridman 1:15:30 ...yeah.

Rodney Brooks 1:15:30 ...is that people make interpolations from that.

Lex Fridman 1:15:33 Interpolations.

Rodney Brooks 1:15:34 That, uh, you know.

Lex Fridman 1:15:34 Yeah.

Rodney Brooks 1:15:34 It's here, it's done. Um, you know, it's just, you know, we've solved it. No we haven't yet. And, and that's my argument, you know.

Lex Fridman 1:15:42 Okay. So I'd like to go to... 'cause it's, you, uh, you keep a list of predictions-

Rodney Brooks 1:15:46 Yep. (laughs)

Lex Fridman 1:15:47 ...on, on your amazing blog post. It'd be fun to go through them. But before them, let me, let me ask you about this. You have a,

Lex Fridman 1:15:55 y- you have a harshness to you sometimes in your criticisms of what is crit-

Rodney Brooks 1:16:01 What? (laughs)

Lex Fridman 1:16:01 See? (laughs) .

Rodney Brooks 1:16:03 (laughs)

Lex Fridman 1:16:03 That's hype. (laughs)

Rodney Brooks 1:16:04 (laughs)

Lex Fridman 1:16:05 Uh, a- and so like, c- 'cause people extrapolate, like you said, and they, they kind of buy into the hype and then they, th- they kind of start to think that, um, uh, the, the technology's way better than it is. But let me ask you maybe a difficult question.

Rodney Brooks 1:16:23 Sure.

Lex Fridman 1:16:25 D- do you think, if you look at history of progress, don't you think to achieve the, quote, "impossible," you have to believe that it's possible?

Rodney Brooks 1:16:34 Oh, abso- absolutely. Yeah. Look, look, his, his, his, his, his two great runs, great, unbelievable, 1903, first human, um, pow- y- human, um, you know, heavier than air flight.

Lex Fridman 1:16:49 Yeah.

Rodney Brooks 1:16:50 1969, we land on the moon.

Lex Fridman 1:16:52 Yeah.

Rodney Brooks 1:16:52 That's 66 years. I'm 66 years old.

Lex Fridman 1:16:55 Yeah.

Rodney Brooks 1:16:55 In my lifetime, that span of my lifetime-

Lex Fridman 1:16:58 Yeah.

Rodney Brooks 1:16:58 ...we went from barely g- you know, flying, I don't know what it was, 50 feet-

Lex Fridman 1:17:02 Yeah.

Rodney Brooks 1:17:02 ...or the length of the first flight or something, to landing on the moon. Unbelievable.

Lex Fridman 1:17:08 Yeah.

Rodney Brooks 1:17:08 Fantastic.

Lex Fridman 1:17:09 But that requires, by the way, one of the Wright brothers, both of them, but one of them didn't believe it's even possible like a year before. Right? So the, like, not just possible soon, but like-

Rodney Brooks 1:17:20 Yeah.

Lex Fridman 1:17:20 ...ever.

Rodney Brooks 1:17:21 So, so, so, so, you know-

Lex Fridman 1:17:22 How important is it to believe and be optimistic is what I guess I'm asking?

Rodney Brooks 1:17:25 Oh yeah, it is important. It's when it goes craz- when, when, when I, you know, you said, uh, what, what did, what was the word you used for my bad...

Lex Fridman 1:17:34 Harshness?

Rodney Brooks 1:17:34 Harshness, yes. (laughs)

Lex Fridman 1:17:41 (laughs)

Rodney Brooks 1:17:41 (laughs) Um.

Lex Fridman 1:17:41 (laughs)

Rodney Brooks 1:17:41 (laughs) I just get so frustrated-

Lex Fridman 1:17:42 Yes.

Rodney Brooks 1:17:43 ...when, when people make these leaps and tell me that I'm, that I don't understand.

Lex Fridman 1:17:49 Right.

Rodney Brooks 1:17:49 I, I, you know, yeah. There's, just from iRobot, which I was co-founder of-

Lex Fridman 1:17:57 Yeah.

Rodney Brooks 1:17:57 ...I don't know the exact numbers now 'cause I haven't, it's 10 years since I stepped off the board, but I believe it's well over 30 million robots cleaning houses.

Lex Fridman 1:18:06 Yeah.

Rodney Brooks 1:18:06 From that one company. And now there's lots of other companies-

Lex Fridman 1:18:08 Yes.

Rodney Brooks 1:18:08 ...but, um, was that a crazy idea that we had to believe, uh, in 2002 when we released it? Yeah. That was, we, we had, we had to, you know, believe that it could be done.

Lex Fridman 1:18:23 Le- let me ask you about this. So iRobot, one of the greatest robotics companies ever in terms of manufac- uh, uh, creating a robot that actually works in the real world, probably the greatest robotics company ever. You were the co-founder of it. Um, if, if the Rodney Brooks of today talked to the Rodney of back then, what would you tell him? 'Cause I have a sense that... Would you pat him on the back and say, "What you're doing is going to fail, but go at it anyway"? That's what I'm referring to with the harshness. You've accomplished an incredible thing there, one of several things we'll talk about. What, w- like that's what I'm trying to get at, that line.

Rodney Brooks 1:19:06 No. It's, it's when, w- eh, my harshness is reserved for people who are not doing it-... who claim it's just, "Well, this shows that it's just going to happen."

Lex Fridman 1:19:16 But here- here's the thing-

Rodney Brooks 1:19:18 This shows-

Lex Fridman 1:19:18 ... but you have that harshness for Elon too.

Rodney Brooks 1:19:23 Um, no.

Lex Fridman 1:19:24 Or no, it's a different harshness?

Rodney Brooks 1:19:26 No, it's, it's a, a different, um, argument with Elon. You know, I, I, I think SpaceX is an amazing company. O- on the other hand, you know, I, in one of my blog posts I said, "What's easy and what's hard?"

Lex Fridman 1:19:40 Yes.

Rodney Brooks 1:19:40 I said, "SpaceX, um, vertical landing rockets, it had been done before." Um, grid fins had been done since the '60s. Every Soyuz has them. Um, um, reusable space, um, DCX reused those rockets that landed vertically. Um, th- there was a whole insurance industry in place for rocket launches. There were all sorts of infrastructure. That was doable. It took a great entrepreneur, a great personal expense, he almost drove himself, you know, bankrupt doing it.

Lex Fridman 1:20:17 Mm-hmm.

Rodney Brooks 1:20:17 Um, a great belief to do it. Whereas Hyperloop, uh, there's a whole bunch more stuff that's never been thought about and never been demonstrated, so my estimation is Hyperloop is a long, long, long-

Lex Fridman 1:20:32 Yes.

Rodney Brooks 1:20:32 ... a lot further off.

Lex Fridman 1:20:33 But as- as-

Rodney Brooks 1:20:34 And, and if I've got a criticism of, of, of Elon, it's that he doesn't make distinctions between when the technology's coming along and ready, and then he'll go off and, and-

Lex Fridman 1:20:46 See, I- I-

Rodney Brooks 1:20:47 ... mouth off about other things, which then people go and compete about and try and do, and so...

Lex Fridman 1:20:54 See, this is where I, um, I, I understand what you're saying. I tend to draw a different distinction. I, I have, uh, a similar kind of harshness towards people who are not telling the truth, who are basically fabricating stuff to make money or to-

Rodney Brooks 1:21:10 Well, he believes what he says. I just think he's wrong sometimes.

Lex Fridman 1:21:12 I, to me, that's a very important difference.

Rodney Brooks 1:21:13 Yeah, I'm not, I'm not...

Lex Fridman 1:21:15 (laughs)

Rodney Brooks 1:21:15 Uh...

Lex Fridman 1:21:15 Because I think, uh, in order to fly, in order to get to the moon, you have to believe, um, even when, uh, most people tell you you're wrong, and most likely you're wrong, but sometimes you're right. I mean, tha- that's the same thing I have with Tesla autopilot. I, I think that's an interesting one. I was, especially when I was, you know, um, at MIT, and just the entire human factors in the robotics community, were very negative towards Elon. It was very interesting for me to observe colleagues at MIT. I wasn't sure what to make of that. That was very upsetting to me, because I understood where tha- where that's coming from, and I, I agreed with them, and I kind of almost felt the same thing in the beginning, until I kind of opened my eyes and, and realized there's a lot of interesting ideas here. There might be over-hype. You know, if- if you focus yourself on the idea that, um, you shouldn't call a system full self-driving when it's obviously not autonomous, fully autonomous, you're going to miss the magic-

Rodney Brooks 1:22:17 Oh.

Lex Fridman 1:22:17 ... of progress there.

Rodney Brooks 1:22:17 Y- yeah, you aren't gonna miss the magic, but at the same time, there are people who buy it, literally pay money for it-

Lex Fridman 1:22:24 Yeah.

Rodney Brooks 1:22:24 ... and take those words as given.

Lex Fridman 1:22:27 So, it's-

Rodney Brooks 1:22:27 And that's-

Lex Fridman 1:22:28 Uh, but I haven't... So, (laughs) the, I, take words as given is one thing. I haven't actually seen people that use autopilot that believe that, that behavior is really important, like the actual action. So like, this is, I, to push back o- on the very thing that you're frustrated about, which is like journalists and general people, uh, buying all the hype and going on, in the same way, I think there's a lot of hype about m- the, the negatives of this too.

Rodney Brooks 1:22:57 Yeah.

Lex Fridman 1:22:57 That people are buying without using. People use, the way... This is what, this was, this opened my eyes actually. The way people use a product is very different than the way they talk about it. This is true with robotics, with everything. Everybody has dreams of how a particular product might be used or so on. There's, and then when it meets reality, there's a lot of fear of robotics, for example. That robots are somehow dangerous and all those kinds of things. But when you actually have robots in your life, whether it's in the factory or in the home making your life better, that's going to be, that's way different. Your perceptions of it are going to be way different. And so my just tension was, was like here's an innovator, um, uh, like, uh, uh, what is it? Sorry, sorry. Super Cruise from Cadillac was super interesting too. That's a really interesting system. There's inno- We should like be excited by those innovations.

Rodney Brooks 1:23:48 Okay, so let me-

Lex Fridman 1:23:48 Yeah.

Rodney Brooks 1:23:48 Can I tell you something that's really annoyed me recently?

Lex Fridman 1:23:51 Mm-hmm.

Rodney Brooks 1:23:51 It's really annoyed me that the press and friends of mine on Facebook are going, "These billionaires and their space games."

Lex Fridman 1:24:00 (laughs)

Rodney Brooks 1:24:00 You know, "Why are they doing that?"

Lex Fridman 1:24:02 Yeah, that's been very frustrating.

Rodney Brooks 1:24:03 And that really, really pisses me off.

Lex Fridman 1:24:04 Yeah.

Rodney Brooks 1:24:04 I, I must say. I, I, I applaud that.

Lex Fridman 1:24:07 Yeah.

Rodney Brooks 1:24:07 I applaud it.

Lex Fridman 1:24:08 Yeah.

Rodney Brooks 1:24:09 It's the taking, and not necessarily the people who are doing the things, but, you know, that I, that I keep having to push back against unrealistic expectations of when these things can become real.

Lex Fridman 1:24:22 Yeah. I, this was interesting on a... Because there's been a particular focus for me is autonomous driving. Elon's prediction of when certain milestones would be hit, um, (sighs) there's several things to be said there that I al- I always, I thought about because whenever he said them, it was obvious that's not going... To me, as a person that kind of not inside the system, it was obvious it's unlikely to hit those. Uh, there's two comments I want to make. One, he legitimately believes it. And two, much more importantly, I think that having ambitious deadlines drives people to do the best work of their life, even when the odds of those deadlines are very, um, low.

Rodney Brooks 1:25:11 To, to a point, and I'm not-

Lex Fridman 1:25:12 To a point.

Rodney Brooks 1:25:12 So I'm not talking about anyone here.

Lex Fridman 1:25:13 Yes.

Rodney Brooks 1:25:13 I'm just saying

Narrator 1:25:14 ... again.

Lex Fridman 1:25:14 So there's a line there, right?

Rodney Brooks 1:25:15 There's, you have to have a line, because you, you over-extend and it's-... it's demoralizing.

Lex Fridman 1:25:21 Yeah.

Rodney Brooks 1:25:23 But I will say that there's an additional thing here, that those words also, um, drive the stock market.

Lex Fridman 1:25:33 Yeah.

Rodney Brooks 1:25:34 And, you know, we have, because of the way that rich people in the past have manipulated the rubes through investment, we have, um, um, we have developed rules about what, you know, what you're allowed to say and-

Lex Fridman 1:25:49 Yeah, I-

Rodney Brooks 1:25:50 ... and your promise. And, you know, there's an area here which is-

Lex Fridman 1:25:54 I, I, uh, I tend to be, uh, maybe I'm naive, but I, I tend to believe, uh, that, um, like engineers, innovators, people like that, they're not, th- they're my, they don't think like that, like manipulating the pri- the stock