Marc Raibert: Boston Dynamics and the Future of Robotics | Lex Fridman Podcast #412

Marc Raibert 0:00 So BigDog became LS3, which is the big load-carrying one.

Lex Fridman 0:03 Just, just a quick pause. It can carry 400 pounds and-

Marc Raibert 0:08 It was designed to carry 400, but we had it carrying about 1,000 pounds, uh, one time. (laughs)

Lex Fridman 0:12 Of course you did. (laughs) Just to make sure.

Marc Raibert 0:15 We had one carrying the other one. We had two of them, so we had one carrying the other one.

Lex Fridman 0:19 So one of the things that stands out about the robots Boston Dynamics have created is how beautiful the movement is, how natural the walking is and r- running is, even flipping is, throwing is. So maybe you can talk about what, what's involved in making it look natural.

Marc Raibert 0:37 Well, I think having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong.

Lex Fridman 0:48 The following is a conversation with Marc Raibert, a legendary roboticist, founder and longtime CEO of Boston Dynamics, and recently, the executive director of the newly created Boston Dynamics AI Institute that focuses on research and the cutting edge, on creating future generations of robots that are far better than anything that exists today. He has been leading the creation of incredible legged robots for over 40 years, at CMU, at MIT, the legendary MIT Leg Lab, and then of course, Boston Dynamics, with amazing robots like BigDog, Atlas, Spot, and Handle. This was a big honor and pleasure for me. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Marc Raibert.

Lex Fridman 1:43 When did you first fall in love with robotics?

Marc Raibert 1:47 Well, I was always a builder from a, from a young age. I wa- I was lucky. My father was a, uh, frustrated, uh, engineer, and by that I mean, uh, he wanted to be an aerospace engineer, but his mom, from the old country, thought that that would be like a grease monkey. (laughs)

Lex Fridman 2:05 Mm-hmm.

Marc Raibert 2:05 And so she said no, so he became an accountant. But the, but the result of that was our basement was always full of, uh, tools and equipment and electronics and, you know, from a young age I would watch him, uh, assembling a kit, an ICO kit or something like that. I still have a couple of his old ICO kits. And, uh, but it was really, uh, during graduate school when, uh, uh, I followed, uh, a professor back, uh, from class. It was, uh, Berthold Horn at MIT, and I was taking a, uh, an interim class. It's IAP, Independent Activities Period.

Lex Fridman 2:42 Mm-hmm.

Marc Raibert 2:43 And I followed him back to his lab, and on the table was a, a Vicarm robot arm taken apart in probably a thousand pieces. And, uh, when I saw that, you know, from that day on, uh, I was a roboticist. (laughs)

Lex Fridman 2:58 Do you remember the year?

Marc Raibert 2:59 1974.

Lex Fridman 3:01 1974. So there's just this arm in pieces-

Marc Raibert 3:04 Yeah.

Lex Fridman 3:04 ...and you, you saw the pieces and you saw in your v- in your vision, the, the, the arm when it's ba- put back together and the possibilities that holds.

Marc Raibert 3:12 Somehow it, uh, it spurred my imagination. I ha- I was in the, uh, Brain and Cognitive Sciences Department as a graduate student doing neurophysiology. I'd been a electrical engineer as an undergrad at Northeastern, and, uh, the neurophysiology wasn't really working for me, you know? I, it wasn't conceptual enough. I couldn't see really how by looking at single neurons you were gonna get to a place where you could understand, you know, control systems or thought or anything like that. And, uh, you know, the AI Lab was always an appealing... This was before CSAIL, right? This was in the '70s. So-

Lex Fridman 3:48 Mm-hmm.

Marc Raibert 3:48 ...the AI Lab was always an appealing idea, and so when I went to the AI Lab with, uh, you know, following him, uh, and I saw the arm, I just thought, you know, "This is it."

Lex Fridman 3:59 It's so interesting, the tension between the, the, the BCS, Brain and Cognitive Science approach to understanding intelligence and the robotics approach to understanding intelligence.

Marc Raibert 4:09 Well, BCS has now morphed a bit, right? They, they have the Center for Brains, Minds, and Machines, which is, uh, trying to bridge that gap. And even when I was there, you know, David Marr was in the AI Lab. David Marr had models of the brain that were appealing both to biologists but also to, uh, computer people. So he was a visitor in the AI Lab at the time, and I guess he became full-time there. So that was the first time a bridge was made between those two groups. Then the bridge kind of went away, and then there was another time in the '80s, and then recently, uh, you know, the last five or so years there's been a, a stronger connection.

Lex Fridman 4:48 You said you were always kind of a builder. What, what, what stands out to you in memory of a thing you've built, maybe a trivial thing that just kind of like inspired, um, inspired you in the possibilities that this direction of work might hold?

Marc Raibert 5:02 I mean, we were just doing gadgets when we were kids-

Lex Fridman 5:04 Yes.

Marc Raibert 5:04 ...you know? I had a friend, we were taking, uh, you know the... I don't know if everybody remembers, but fluorescent lights had this, uh, little, m- uh, aluminum cylinder. Uh, and I can't even remember what it's called now, that you needed in a starter I think it was.

Lex Fridman 5:20 Mm-hmm.

Marc Raibert 5:20 Uh, and we would take those apart, fill them with match heads, put a tail on it, and make it into little rockets. (laughs)

Lex Fridman 5:27 So it wasn't always about function. It was... Well-

Marc Raibert 5:30 Well, rocket was pretty-

Lex Fridman 5:31 (laughs)

Marc Raibert 5:31 ...pretty much functional.

Lex Fridman 5:31 (laughs) I guess that is pretty functional.

Marc Raibert 5:33 Yeah.

Lex Fridman 5:33 But, I- uh, I guess that is a question. How much was it about function versus just creating something cool?

Marc Raibert 5:40 I think it's a, it's still a balance between those two.

Lex Fridman 5:42 Hm.

Marc Raibert 5:43 There was a time though when I was a, a- I guess I was probably already a professor or maybe late in graduate school when I thought that function was everything and that, uh, you know, mobility, dexterity, perception, and intelligence, those are sort of the functio- the key functionalities for robotics, that, um...... that's what mattered, and nothing else mattered. And I even had, uh, kind of this platonic ideal, uh, that a robot, if you just looked at a robot and it wasn't doing anything, it would look like a pile of junk, which a lot of my robots looked like in thos- in those days. But then when it started moving, you'd get the idea that it, you know, it had some kind of life or some kind of, uh, interest in its movement. And I think we purposely even designed the machines not, not worrying about the aesthetics of the, of the structure itself. Uh, but then, uh, but then, you know, it turns out that the aesthetics of the thing itself add and combine with, uh, the lifelike things that the robots can do. But the heart of it is, you know, making them do things that are, that are interesting.

Lex Fridman 6:47 So one of the things that underlies a lot of your work is that the robots you create, the systems you have created for, for over 40 years now have a kind of, they're not cautious. So a lot of robots that people know about move about this world very cautiously, carefully, very afraid of the world. Uh, a lot of the robots you built, especially in the early days, were very aggressive, uh, under-actuated. They're hopping. They're, (laughs) they're wild, moving quickly. So what, is there a philosophy underlying that?

Marc Raibert 7:20 Well, let me tell you about how I got started on legs at all. I, when I was still a graduate student, I went to a conference. It was a biological legged locomotion conference, and I think it was in Philadelphia. So it was all biomechanics people. You know, researchers who would look at muscle and maybe neurons and things like that. They weren't so much computational people, but they were more biomechanics. And maybe there were a thousand people there. And I went to a talk. Uh, one of the talks, all the talks were about the body of either animals or people and respiration, things like that. But one talk was by a robotics guy.

Lex Fridman 7:56 Mm-hmm.

Marc Raibert 7:56 And he showed a six-legged, uh, robot that walked very slowly. Um, it always had at least three feet on the ground, so it worked like a table or a chair with tripod stability, and it moved really slowly. And I just looked at that and said, "Wow, that's wrong." You know, that's not, that's not anything like how people and animals work because we balance and fly, you know, we have to predict what's gonna happen in order to keep our balance when we're taking a running step or something like that. We use the springiness in our, in our legs, you know, in our muscles and our tendons and things like that as part of the story, you know, the energy circulates. We don't just throw it away-

Lex Fridman 8:38 Mm-hmm.

Marc Raibert 8:38 ... every time. So no, I'm not sure I understood all that when I first thought, but I, I definitely got inspired to say, you know, "Let's try the opposite." And I didn't have a clue as to how to make a hopping robot work, not re- you know, not balanced in 3D. Uh, in fact, when I started, it was all just about the energy of bouncing and I was gonna have a springy thing in the leg and some actuator so that you could get an energy, uh, regime going of bouncing. And the idea that balance was an important part of it didn't come until a little later. Uh, and then, you know, I made the, the one-legged, uh, the pogo stick robots.

Lex Fridman 9:16 Mm-hmm.

Marc Raibert 9:17 Now I think that we need to do that in manipulation. If you look at robot manipulation, we've been working, we, a community has been working on it for 50 years. We're nowhere near human levels of manipulation. I mean, we can, you know, it's come along, but I think it's all too safe, and I, I think trying to break out of that safety thing of static grasping. You know, if you look at the, a lot of work that goes on, it's about the geometry of the part and then, and then you figure out how to move your hands so that you can position it with respect to that, and then you grasp it carefully, and then you move it. Well, that's not anything like how people and animals work, you know? We juggle in our hands. We hug multiple objects and can sort them. Um, so now, to be fair, uh, being more aggressive is gonna mean things aren't (laughs) gonna work very well for a while. So it's a long, it's a longer term approach to the problem. Um, but that, and that's just theory now. You know, maybe that won't pay off, but that's sort of how I'm trying to think about it, trying to, uh, encourage our group to, to go at it.

Lex Fridman 10:22 Well, yeah, I mean, we'll talk about what it means to-

Marc Raibert 10:25 (laughs)

Lex Fridman 10:26 ... what, what is the actual thing we're trying to optimize and, uh, for a robot. You know, sometimes especially with human robot interaction, maybe flaws is a good thing. Uh, perfection is not necessarily the right thing to be chasing. Just like you said, maybe, maybe being good at fumbling an object, uh, being good at fumbling might be the right thing to optimize versus perf- perfect modeling of the object and perfect movement of the arm to gras- grasp that object 'cause, uh, maybe perfection is not supposed to exist in the real world.

Marc Raibert 10:57 I don't know if you know my friend Matt Mason who's, uh, who is the, uh, director of the Robotics Institute at Carnegie Mellon and we go back to graduate school together. But he analyzed, um, a movie of Julia Childs doing a cooking thing, and she did, I think he said something like there were 40 different ways that she handled a thing and none of 'em was grasping. He would, she would nudge, roll, flatten with her, you know, knife, things like that, and none of them was grasping.

Lex Fridman 11:28 (laughs) So okay, let's go back to the early days. First of all-

Marc Raibert 11:30 Sure.

Lex Fridman 11:31 ... you've, you've created and led the Leg Lab, the legendary Leg Lab at MIT. So what, what was that first hopping robot? Can you...

Marc Raibert 11:38 But first of all, the Leg Lab actually started at Carnegie Mellon.

Lex Fridman 11:41 Carnegie Mellon.

Marc Raibert 11:42 So I was a professor there starting, uh, in 1980, uh, into about 1986. And, uh, so that's where the first hopping machines were built, uh, starting, I guess we got the first one working in about 1982, something like that. That was a simplified one. Then we got a three-dimensional one in 1983.... the quadruped that, uh, we built at the Leg Lab, the first version, was built in about 1984 or '5, and really only got going about '86 or so. And it took years of development to get it to-

Lex Fridman 12:17 Let's just pause here.

Marc Raibert 12:18 (laughs)

Lex Fridman 12:18 For people who don't know, I'm talking to Marc Raibert, founder of Boston Dynamics. But before that, you were a professor developing some of the most incredible robots for 15 years. And before that, of course, a grad student and all that. So you've been doing this for a really long time. So we- you, like, skipped over this, but, like, go, go to the first hopping robot. There's videos of some of this. I mean, these are incredible robots that you talked about. The fir- uh, th- th- the very first step was to get a thing hopping up and down.

Marc Raibert 12:46 Right.

Lex Fridman 12:47 And then you realized, well, balancing is a thing you should care about, and it's actually a solvable problem. So j- can you just go through how to create that robot? What was- what-

Marc Raibert 12:57 Sure.

Lex Fridman 12:57 ... what was involved in creating that robot?

Marc Raibert 13:00 Well, I'm gonna start on the- not the technical side-

Lex Fridman 13:03 Mm-hmm.

Marc Raibert 13:03 ... but the, uh, I guess we could call it the motivational side-

Lex Fridman 13:07 (laughs) Sure.

Marc Raibert 13:07 ... or the funding side.

Lex Fridman 13:08 (laughs)

Marc Raibert 13:08 So before Carnegie Mellon, I was actually at JPL, at the Jet Propulsion Lab, for three years. And while I was there, I connected up with, uh, Ivan Sutherland, who is sometimes regarded as the father of computer graphics 'cause of work he did both at MIT and then University of Utah and Evanston-Sutherland. Anyway, um, I got to know him, and at one point he said, uh, he encouraged me to, uh, do some kind of project, uh, at Caltech, even though I was at JPL. You know, those are kind of related institutions. And, uh, so I, I thought about it, uh, and I made up a list of three possible projects.

Lex Fridman 13:51 Mm-hmm.

Marc Raibert 13:51 And I purposely made the top one and the bottom one really boring sounding, and in the middle, I put pogo stick robot. And when he looked at it, you know, Ivan is a, a brilliant, uh, f- you know, brilliant guy, brilliant engineer, and a real cultivator of people. He looked at it and knew right away what the thing that was worth doing. And so he, you know, he had an endowed chair, so he had about $3,000 that he gave me to build the first model, which I went, you know, I went to the shop and with my own hands kind of made a first model, which, which didn't work, uh, and was just, you know, a, a beginning, uh, shot at it. And, uh, Ivan and I took that to Washington, and in those days you could just walk into DARPA and walk down the hallway and see who's there.

Lex Fridman 14:40 Yeah.

Marc Raibert 14:40 And Ivan, who had been there in his previous life, and so we walked around and, uh, we looked in the offices 'cause I didn't know anything, you know, I was basically a kid, but Ivan knew his way around, and we found Craig Fields, uh, in his office. Craig later became the director of DARPA, but in those days he was the program manager.

Lex Fridman 14:58 Mm-hmm.

Marc Raibert 14:58 And so we went in, I had a little Samsonite suitcase, we, we opened, and it had just the skeleton of this, uh, one-legged hopping robot, and we showed it to him, and, uh, you could almost see the drool going down his chin-

Lex Fridman 15:11 'Cause he was excited?

Marc Raibert 15:11 ... of excitement. And he sent me $250,000. He said, "Okay, uh, I'll, uh, I wanna fund this." Uh, and I was between institutions. I was just about to leave JPL, and I hadn't decided yet where I was going next. And then when I landed at CMU, he sent $250,000, which in 1980 was a lotta, a lotta research money.

Lex Fridman 15:34 Did you see the possibility of where this is going, why this is an important problem?

Marc Raibert 15:38 (laughs) No.

Lex Fridman 15:39 The balancing, I mean, it's legged, it, it has to do with leg locomotion. I mean, it has to do with all these problems that, that, that are, th- the human body solves when we're walking, for example. Like, all the fundamentals are there.

Marc Raibert 15:51 Yeah, I mean, I think that was the motivation, to try and get more at the fundamentals of how animals work. But the idea that it would result in, you know, machines that were anything like practical, uh, like we're making now, that- that wasn't anywhere in my head, no. You know, as an academic, I was mostly just trying to do the next thing, you know, make some progress, impress my colleagues if I could.

Lex Fridman 16:14 And have fun.

Marc Raibert 16:15 And have fun.

Lex Fridman 16:16 Pogo stick robot.

Marc Raibert 16:17 Pogo stick robot.

Lex Fridman 16:18 So what was, on the technical side, what are the, some of the challenges of getting up- getting to the point where we saw, like in the video, the, the pogo stick robot that's actually successfully hopping and then eventually doing flips and all this kind of stuff?

Marc Raibert 16:31 Well, in the very early days, I needed some better engineering than I had, than I could do myself, and I hired, uh, Ben Brown. And we, we each had our way of contributing to the design, and we came up with a thing that could, could start to work. I had some stupid ideas about how the actuation system should work, and, uh, we, you know, we sorted that out. It wasn't that hard to make it balance once you get the, the physical machine to be working well enough, uh, and have enough control over the degrees of freedom. Uh, and then we very quickly, you know, we started out by having it floating on a inclined air table, and then, uh, that only gave us, like, six foot of travel. So once it started working, we switched to a thing that could run around the room on a- another device. It's hard to explain these without you seeing them, but y- you probably know what I'm talking about-

Lex Fridman 17:20 Mm-hmm.

Marc Raibert 17:21 ... a planarizer. And, uh, and then the next big step was to make it work in 3D, which that was really the scary part, were these simple things. You know, people had inverted pendulums at the time for, for years and they could control them by driving a cart back and forth. But could you make it work in three dimensions while it's bouncing and all that? And, uh, but it turned out, you know, not to be that hard to do, uh, at least at the level of performance we achieved at the time.

Lex Fridman 17:46 So, okay, you mentioned inverted pendulum, but, like, uh, can you explain how a hopping stick in 3D can control- can balance itself?

Marc Raibert 17:58 Yeah, sure.

Lex Fridman 17:58 What does- what does the actuation look like?

Marc Raibert 18:01 Uh, you know, the simple story is that there's three things going on. There's something making it bounce, and, you know, we, we had a system that was, uh...... estimating how high the robot was off the ground, and using that, you know, uh, there's energy that can be in three places in a, in a pogo stick. One is in the spring, one is in the altitude, and the other is in the velocity. And so when at the top of the hop, it's all in the v- the, uh, height, and so you could just measure how high you're going and thereby, thereby have an idea of a lot about the cycle, and you could decide whether to put more energy in or less. So that is one element. Then there's the part that you decide where to put the foot, and if you think, when you're landing on the ground, with respect to the center of mass. So if you think of a pole vaulter, the key thing the pole vaulter has to do is get its body to the right place when the pole gets stuck. If they're too far forward, uh, they kind of get thrown backwards. If they're too far back, they go, you know, over, and what they need to do is get it so that they go mostly up to get over the thing. And you know, high jumpers is the same kind of thing. So there's the calculation about where to put the foot, and we did something, you know, relatively simple. And then there's a third part to keep the body at an attitude that's upright, 'cause if it gets too far, you know, you could hop and just-

Lex Fridman 19:23 (laughs)

Marc Raibert 19:23 ... keep rotating around, but if it gets too far, then you r-run out of motion of the joints at the hips, so you have to do that. And we did that by applying a torque between the legs and the body every time the foot's on the ground. You only can do it while the foot's on the ground.

Lex Fridman 19:38 Mm-hmm.

Marc Raibert 19:38 In the air, you know, it, it, it, the physics don't work out.

Lex Fridman 19:42 How far does it have to tilt before it's too late to be able to balance itself? Or it's impossible to balance itself, correct itself?

Marc Raibert 19:50 Well, you're, you're asking an interesting question because, um, in those days, we didn't actually optimize things, and they probably could have gone much further than we did and then had higher performance, and we just kind of got, you know, a sketch of a solution and worked on that. And then y- in years since, some people working for us, some people working for others, people came up with all kinds of, uh, equations for, or, you know, algorithms for how to do a better job, be able to go faster. Uh, one of my students worked on getting things to go faster, another one worked on, uh, climbing over obstacles, 'cause when you're running, it's one, on the open ground it's one thing. If you're running, like, up a stair, uh, you have to adjust where you are. Otherwise, things don't work out right. You land your foot on the edge of the steps. So there's other degrees of freedom to control if you're getting to, you know, more realistic, practical situations.

Lex Fridman 20:44 I think it's really interesting to ask about the early days, 'cause, you know, believing in yourself, believing that there's something interesting here, and then you mentioned findi-finding somebody else, Ben Brown. What's that like, finding other people with whom you can build this crazy idea and actually make it work?

Marc Raibert 21:00 Probably the smartest thing I ever did-

Lex Fridman 21:02 (laughs)

Marc Raibert 21:02 ... is to find the other people.

Lex Fridman 21:04 Yeah.

Marc Raibert 21:04 I mean, when I look at it now, you know, I look at Boston Dynamics and all the really excellent engineering there, you know, people who really make stuff work, uh, you know, I'm, I'm only the, the dreamer.

Lex Fridman 21:16 So when you talk about pogo stick robot or legged robots, whether it's quadrupeds or humanoid robots, did people doubt that this is possible? Did you experience a lot of people around you kind of-

Marc Raibert 21:29 I don't th- I don't know if they doubted whether it was possible, but I think they thought it was a waste of time. (laughs)

Lex Fridman 21:34 Oh, it was not even an interesting problem, there-

Marc Raibert 21:36 I think for a lot of people, you know, people who were... Eh, I think it's been, it's been both though. Some people, I think, I felt like they were saying, "Oh, you know, why are you wasting your time on this stupid problem?" And then, but then, I've been at many things where, uh, people have told me it's been an inspiration to, uh, to go out and, uh, you know, attack these, uh, these harder things. And, and I think it has turned out... I think legged locomotion has turned out to be a useful thing.

Lex Fridman 22:06 Did you ever have, like, doubt about bringing Atlas to life, for example? Or, or, or with BigDog, just every step of the way, did you have doubt, like, "Wh- wh- this is, this is too hard of a problem?"

Marc Raibert 22:19 I mean, at first I wasn't an enthusiast for the humanoids, 'cause again, it goes back to saying, "What's the functionality?" And the form wasn't as important as the functionality, uh, and I th- and also, you know, there, there's a, an aspect to humanoid robots that's about, uh, all about the cosmetics, where there isn't really other functionality, and that kind of is off-putting for me, uh, as a roboticist. I think the functionality really matters. So probably that's why I avoided, uh, human robots, humanoid robots to start with. But I'll tell you, um, now, you know, after we started working on them, you could see that the imp- the connection and the impact with, with, uh, other people, whether they're laypeople or even other technical people, uh, there's a, there's a, a special thing that goes on-

Lex Fridman 23:08 Mm-hmm.

Marc Raibert 23:08 ... uh, even though most of the humanoid robots aren't that much like a person.

Lex Fridman 23:11 But we anthropomorphize and we see-

Marc Raibert 23:13 (laughs) That's right.

Lex Fridman 23:13 ... the humanity. Uh, but also, like, with, uh, with Spot, you can see not the humanity, but the, whate- whatever we find compelling about social interaction is there in Spot as well.

Marc Raibert 23:25 I tell you, you know, I go around giving talks and take Spot to, to a lot of them.

Lex Fridman 23:29 Mm-hmm.

Marc Raibert 23:29 And it's amazing. The media likes to say that they're terrifying and, and that people are afraid, and, and YouTube commentators like to say that it's frightening, but when you take a Spot out there y- a- now, maybe it's self-selecting, but you get a crowd of people who want to take pictures, want to pose for selfies, want to operate the robot, want to pet it, want to put clothes on it. Uh, it's amazing.

Lex Fridman 23:52 Yeah, I love Spot. So if we, uh, move around history a little bit, so you said, I think in the early days of Boston Dynamics, that you quietly worked on making, uh, a running version of Aibo.

Marc Raibert 24:04 Yeah, yeah.

Lex Fridman 24:04 Sony's robot dog.

Marc Raibert 24:05 Yeah.

Lex Fridman 24:06 It's just an interesting, uh-

Marc Raibert 24:07 (laughs)

Lex Fridman 24:08 ... little tidbit of history for me. Um-What, what, what, like, what stands out to your memory from that task? For, for people who don't know, that little dog robot moves slowly. How did that become big dog? What was involved there? What was the dance between how do we make this cute little dog versus a thing that can actually carry a lot of payload and move fast and stuff like that?

Marc Raibert 24:29 What the connection was is that at that point, Boston Dynamics was mostly a physics-based simulation company. So when I left MIT to start Boston Dynamics, you know, there was a few years of overlap, but the concept wasn't to start a robot company. The concept was to use this, uh, dynamic simulation tool that we developed to do robotics to, for other things. Uh, but working with Sony, we got back into robotics, uh, by doing the AIBO Runner, by, we programmed, we made some tools for programming Qrio, uh, which was a small, a humanoid this big, uh, that could do some dancing and, and other kinds of fun stuff. And I don't think it ever reached the market even though they, they did show it. Um, you know, when I look back, I say that we got us back where we belonged.

Lex Fridman 25:15 Yeah. You rediscovered the soul of the company.

Marc Raibert 25:17 That's right.

Lex Fridman 25:18 And so from there, it was always about robots.

Marc Raibert 25:21 Yeah.

Lex Fridman 25:21 (inhales deeply) Uh, so you started Boston Dynamics in 1992.

Marc Raibert 25:27 Right.

Lex Fridman 25:27 Uh, what are some fond memories from the early days?

Marc Raibert 25:31 Uh, one of the robots that we built wasn't, wasn't actually a robot. It was a surgical simulator, but it had force feedback, so it had all the techniques of robotics. And you look down into this, uh, mirror, it actually was, and it looked like you were looking down onto the body you were working on. Your hands were underneath the mirror, so they were where you were looking, and you had tools in your hands that were connected up to these force feedback devices made by, uh, another MIT spinout, Sensible Technologies. So they made the force feedback device, we attached the tools, and we wrote all the software and did all the graphics. So we had 3D computer graphics. It was in the old days when, this was in the late '90s, when you had, uh, a silicon graphics computer that was about this big. Uh, you know, it was the heater in the office, basically.

Lex Fridman 26:22 Nice. Right.

Marc Raibert 26:23 And, uh, and we were doing, uh, surgical operations, anastomosis, which was stitching tubes together, you know, tubes like blood vessels or other things in their body. And you could feel and you could see the tissues move, and it was really exciting. And the idea was to make a trainer to teach surgeons how to do stuff. We built a scoring system because we interviewed, uh, surgeons that told us, you know, what you're supposed to do and what you're not supposed to do. You're not supposed to tear the tissue. You're not supposed to touch it in any place except for where you're trying to engage... There were a bunch of rules. So we built this thing and took it to a trade show, uh, a surgical trade show, and the surgeons were practically lined up. Well, we, we kept a score and we posted their scores like on a video game. And those guys are so competitive that they really, uh, really love doing it. And they would come around and they see someone's score was higher there, so they would come back. But we figured out shortly after that we thought surgeons were gonna pay us to get trained on these things, and the surgeons thought we should pay them in order to, uh, so they could teach us about the thing. And there, there was no money from the surgeons.

Lex Fridman 27:33 Yeah.

Marc Raibert 27:34 And we looked at it and thought, "Well, maybe we could sell it to hospitals that would teach, train their surgeons." And then we said, "Well, we're this..." At the time, we were probably a 12-person company or maybe 15 people, I don't remember. Uh, you know, there's no way we could go after a marketing activity. You know, the company was all bootstrapped in those years. We, we never had investors until Google bought us in, which was after 20 years. So we didn't have any resources to, uh, to go after hospitals. So we at one, sort of at one day, Rob and I were looking at that and we said, uh... We'd built another simulator for knee arthroscopy, and we said, "This isn't gonna work." And we killed it and we moved on, and that was really a milestone in the company because we, you know, we sort of understood who we were and, uh, and what would work and what wouldn't, even though technically it was really a fascinating thing.

Lex Fridman 28:25 What was that meeting like? Were you just, like, sitting at a table, "You know what?" (laughs)

Marc Raibert 28:30 Probably.

Lex Fridman 28:30 "We're going to pivot completely. We're going to let go of this thing we put so much hard work into and then go back to the thing that came from it."

Marc Raibert 28:39 It just always felt right whilst we did it, you know.

Lex Fridman 28:42 Just look at each other and said, "Let's, let's build robots."

Marc Raibert 28:44 Yeah.

Lex Fridman 28:45 What was the first robot you built under b- the, the flag of Boston Dynamics? BigDog?

Marc Raibert 28:51 Well, there was the AIBO, uh, Runner, but it wasn't even a whole robot. It was just legs that we... We took off the legs on AIBOs and attached legs we'd made. And, um, you know, we got that working and showed it to the Sony people. Uh, we worked pretty closely with Sony in those years. One of the interesting things is that, uh, it was before the internet and Zoom and anything like that. So we had six ISDN lines installed, and we would have a telecon every week that worked at very low frame rate, something like 10 hertz. Uh, you know, English across the boundary with, uh, Japan was a challenge, trying to understand what, what each of us was saying and have meetings every week, uh, for, for several years doing that. And, uh, it was a pleasure working with them. They were really supporters. They, they seemed to like us and what we were doing. That was the real transition from us being a simulation company into being a robotics company again.

Lex Fridman 29:51 It was a quadruped? The, the legs were four legs-

Marc Raibert 29:54 Yeah.

Lex Fridman 29:54 ... or two legs?

Marc Raibert 29:55 Yeah, no, four legs. Yeah.

Lex Fridman 29:56 And what did you learn from that experience of, uh, building a, basically a fast-moving quadruped?

Marc Raibert 30:03 Mostly, we learned that something that small, uh, doesn't look very exciting when it's running. It's like it's scampering, and you had to, you had to watch a slow-mo for it to look like it was interesting. If you watch it fast, it was just like a, like-

Lex Fridman 30:17 That's funny.

Marc Raibert 30:18 One of my things was to show stuff in video even from the very early days of the hopping machines. Um...... and so I was always focused on how's this gonna look through the viewfinder. And, uh, running AIBO didn't look so cool through the viewfinder.

Lex Fridman 30:32 So, uh, what- what came next in terms of, uh... What was a big next milestone in terms of a robot you built?

Marc Raibert 30:40 I mean, you gotta say that BigDog was... You know, sort of put us on the map and got our heads really pulled together. We scaled up the company. BigDog was the result of, uh, Alan Rudolph at DARPA, uh, starting a biodynotics program. And he put out a, you know, a request for proposals and, uh, I think there were 42 proposals written and three got funded. One was BigDog, one was a climbing robot, RISE... And, you know, that put things in motion. We, we hired, uh, Martin Buehler. He was a, a professor at Mon- in Montreal at, uh, McGill. He was, uh, incredibly important for getting BigDog, uh, out of the lab and into the mud, which is a... You know, it was a key step to really be willing to go out there and, uh, and build it, break it, fix it, which is sort of one of our mottos at the company.

Lex Fridman 31:32 So testing it in the real world.

Marc Raibert 31:33 Testing.

Lex Fridman 31:33 For people, for people who don't know, BigDog, maybe you can correct me, but it's a, it's a big quadruped, four-leg robot that... It looks big, could probably carry a lot of weight. Not the most weight that Boston Dynamics-

Marc Raibert 31:47 No.

Lex Fridman 31:47 ... have built, but a lot.

Marc Raibert 31:48 Well, it's the first thing that worked. So let's see, if we go back to the leg lab. We'd built a quadruped that could do many of the things that BigDog did, but it had, uh, a hydraulic pump sitting in the room with hoses connected to the robot.

Lex Fridman 32:00 Mm-hmm.

Marc Raibert 32:01 It had a VAX computer in the next room. It needed its own room 'cause it was this giant thing with air conditioning, and it had this very complicated, uh, bus connected to the robot. And the robot itself just had the actuators, it had gyroscopes for sensing and other, some other sensors, uh, but all the power and computing was off-board. BigDog had all that stuff integrated, uh, on the platform. It had a gasoline engine for power, which was a very complicated thing to, to undertake. It had to convert the rotation of the engine into hydraulic power, which is how we, uh, actuated, uh, it. So there was a lot of learning just on the, uh, you know, building the physical robot and, and, uh, the system integration, uh, for that. And then there was the controls, uh, of it.

Lex Fridman 32:49 So for BigDog, you brought it all together onto one platform-

Marc Raibert 32:52 Right.

Lex Fridman 32:53 ... and then so you could, you can-

Marc Raibert 32:54 You could take it out in the woods.

Lex Fridman 32:55 Yeah, you could... And you did. (laughs)

Marc Raibert 32:57 We did. We spent a lot of time down at the, uh, Marine Corps base in Quantico where there was a trail called, uh, the Guadalcanal Trail. And our, uh, milestone that DARPA had specified was that we could go on this one particular trail that involved, you know, a lot of challenge. And we spent a lot of time, our team spent a lot of time down there.

Lex Fridman 33:19 Hi-

Marc Raibert 33:19 Those were fun days.

Lex Fridman 33:20 ... hiking with the robot.

Marc Raibert 33:21 Hiking, yeah.

Lex Fridman 33:21 So what, what did you learn about, like, what it takes to balance a robot like that on a trail? On a hiking trail in the woods? Basically, forget the woods, just the real world. That's the big leap into testing in the real world.

Marc Raibert 33:35 Yeah. As challenging as the woods were-

Lex Fridman 33:38 Mm-hmm.

Marc Raibert 33:39 ... working inside of a home or in an office is really harder. (laughs)

Lex Fridman 33:45 Yeah.

Marc Raibert 33:45 Because when you're in the woods, you can actually take any path up the, up the hill. All you have to do is avoid the obstacles. You c- there's no such thing as damaging the woods, at least, you know, to first order. Whereas if you're in a house, you can't leave scuff marks. You can't bang into the walls. The robots aren't very comfortable bumping into walls, especially in the early days. So I think those were actually bigger challenges once, once we faced them. Uh, it was mostly, you know, getting the systems, uh, to work well enough together, the, the hardware systems to work, and, and the controls. In those days, we did have a human operator who did all the visual perception, uh, going up the Guadalcanal Trail. So, you know, there was a operator who was right there who was very skilled at... Even though the robot was balancing itself and placing its own feet, uh, if the operator didn't do the right thing, it wouldn't go. But years later, we went back with, uh, one of the electric, the pr- precursor to Spot, and, uh, we had advanced the controls and everything so much that, uh, an amateur, complete amateur, could operate the robot f- the first time up and down and up and down, whereas it'd taken us years to, to get there in the previous robot.

Lex Fridman 34:55 So if you fast-forward, BigDog eventually became Spot.

Marc Raibert 34:59 So BigDog became LS3, which is the big load-carrying one.

Lex Fridman 35:03 Just, just a quick pause. It can carry 400 pounds and can-

Marc Raibert 35:07 It was designed to carry 400, but we had it carrying about 1,000 pounds, uh, one time. (laughs)

Lex Fridman 35:12 Of course you did. (laughs) Just to make sure.

Marc Raibert 35:14 We had one, we had one carrying the other one. We had two of them.

Lex Fridman 35:16 (laughs)

Marc Raibert 35:16 So we had one carrying the other one. There's a little clip of that. We should put that out somewhere.

Lex Fridman 35:20 Yeah.

Marc Raibert 35:20 That's from, like, 20 years ago, but...

Lex Fridman 35:22 Wow. Wow. And it, it can go for very long distances. It can travel 20 miles.

Marc Raibert 35:28 Yeah.

Lex Fridman 35:29 Uh...

Marc Raibert 35:29 Gasoline.

Lex Fridman 35:30 Gasoline, yeah. And that, that adventures... Okay, sorry. So LS3 then, what... Uh, how did that lead to Spot?

Marc Raibert 35:38 So BigDog and LS3 had, uh, engine power and hydraulic actuation. Then we made, uh, a robot that was electric, uh, power. So there's a battery driving a motor driving a pump, but still hydraulic actuation.

Lex Fridman 35:58 Mm-hmm. Yeah.

Marc Raibert 35:59 Larry sort of asked us, "Could you make something that weighed 60 pounds that would not be so intimidating if you had it in a house, uh, where there were people?" And that was the inspiration behind, uh, the Spot, pretty much as it exists today. We did a prototype the same size that was the first all-electric, um, non-elec- non-hydraulic robot.

Lex Fridman 36:19 What was the conversation with Larry Page like about... So h- h- here's a guy that kinda is very product-focused and can, can see a vision for, like, what the future holds. That's just interesting kinda aside...... what was the brainstorm about the future of robotics with him like?

Marc Raibert 36:35 I mean, it was almost as simple as what I just said. He, we were, you know, we were having a meeting-

Lex Fridman 36:39 Make it smaller? (laughs)

Marc Raibert 36:39 ... he said, "Yeah, geez, could, you know, do you think you could make a smaller one that wouldn't be so intimidating? You know, like a big dog-"

Lex Fridman 36:45 Yeah.

Marc Raibert 36:45 ... uh, "if it was in your house?" And I said, "Yeah, we could do that." And we started and, and

Marc Raibert 36:52 did.

Lex Fridman 36:52 Is there a lot of technical challenges to go from hydraulic to electric?

Marc Raibert 36:57 You know, I had been in love with hydraulics and still, uh, love hydraulics. Uh, you know, it's, it's a great technology. It's too bad that the, somehow the, the world out there looks at it like it's old-fashioned or that it's, um, icky. And it's true that you do, it is very hard to keep it from having some amount of dripping from time to time. Uh, but if you look at the performance, uh, you know, how strong you can get in a lightweight package... And of course we did a huge amount of innovation. Most of hydraulic, uh, control, that is the valve that controls the flow of oil, had been designed in the '50s for airplanes.

Lex Fridman 37:35 Mm-hmm.

Marc Raibert 37:36 It had been made robust enough, safe enough that you could count on it so that humans could fly in airplanes, and very little innovation had happened. You know, that might not be fair to the people who make the valves, I'm sure that they did innovate, but the basic design had stayed the same. And there was so much more you could do. And so our engineers designed valves, uh, the ones that are in, uh, uh, in Atlas, for instance, that had new kinds of circuits. They sort of did some of the computing that could get you much more efficient use. They were much smaller and lighter, so the whole robot could be smaller and lighter. Uh, we made a hydraulic power supply that had a bunch of components integrated in this tiny package, it's about this big, you know, the size of a football, weighs five, uh, kilograms, and it produces five kilowatts of power. Of course, it has to have a battery, uh, operating, but it's got a motor, a pump, filters, heat exchanger to keep it cool, some valves, all of it, all in this tiny little package. So hydraulics, you know, could still have a ways to go.

Lex Fridman 38:44 One of the things that stands out about the robots Boston Dynamics have created is how beautiful the movement is, how natural the walking is and r- running is, even flipping is, throwing is. So maybe you can talk about what, what's involved in making it look natural?

Marc Raibert 39:02 Well, I think having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong, in my opinion. Um, so I think one of the things certainly in the early years, for me, taking a dynamic approach where you think about what's the evolution of the motion of the thing gonna be, uh, in the future and having a prediction of that that's used at the time that you're giving signals to it, as opposed to it all being servoing, which is, servoing is sort of backward looking. It says, "Okay, where am I now? I'm gonna, I'm gonna try and adjust for that." But you really need to think about what's coming.

Lex Fridman 39:40 So how far ahead do you, do you have to look in time?

Marc Raibert 39:43 Uh, it's interesting. I think that the number is only a couple of seconds for Spot, so there's a limited horizon, uh, type approach where you're recalculating, assuming what's gonna happen in the next, uh, second or second and a half, and then you keep iterating, you know, the next, even though a tenth of a second later you'll say, "Okay, let's do that again and see what's happening." And you're looking at what the obstacles are, where the feet are gonna be placed, how to... You know, you have to coordinate a lot of things if you have obstacles and you're balancing at the same time, and it's that, uh, limited horizon type calculation that's doing a lot of that. But if you're doing something like a somersault, you're looking out a lot further, right? If you wanna stick the landing-

Lex Fridman 40:26 Yeah.

Marc Raibert 40:26 ... you have to get the ro- you know, you have to, at the time of launch, have, uh, you know, momentum and, uh, uh, rotation, all those things coordinated so that a landing is within reach.

Lex Fridman 40:38 How hard is it to stick a landing? I mean, that's, uh, very much underactuated. Like, you, once you've in the air, you don't have as much control about anything. So how hard is it to get that to work? You, first of all, did flips with a hopping robot.

Marc Raibert 40:57 If you look at the first time we ever made a robot do a somersault, it was in a planar robot. You know, it had a boom, uh, so it could only, it was, uh, restricted to the surface of a sphere. We call that planar. So it could move fore and aft, it could go up and down, and it could rotate. And so the calculation of what you need to do to get a, to stick a landing isn't all that complicated. You have to look at, you know, you have to get time to make the rotation, so how hard you jump, how high you jump gives you time. Uh, you look at how quickly you can rotate, and so, you know, if you get those two right, then when you land, you have the feet in the right place, and you have to get rid of all that rotational and, uh, linear momentum. But, you know, that's not too hard to figure out. And we made, you know, back in, uh, about 1985 or '6, I can't remember, we had a simple robot doing somersaults. To do it in 3D, really the calculation is the same, you just have to be balancing in the other degrees of freedom. If you're just doing a somersault, it's just a, a planar thing. Roy Murabawa is my graduate student when we were at MIT, which is when we made a, you know, a two-legged robot do a 3D somersault for the first time. Um, there, we, in order to get enough rotation rate, you needed to do tucking also. Uh, you know, we'd draw the legs in order to accelerate it, and he did some really fascinating work on, on how you stabilize more complicated maneuvers. You remember, he was a gymnast, a champion gymnast before he'd come to me.

Lex Fridman 42:25 (laughs)

Marc Raibert 42:25 So he had, he had the physical abilities, and he was, uh, you know, an engineer, so he could translate some of that into the math and the algorithms that you need to...... to do that.

Lex Fridman 42:37 He knew how humans do it, he just-

Marc Raibert 42:39 Yeah.

Lex Fridman 42:39 ... had to get robots-

Marc Raibert 42:40 Yeah.

Lex Fridman 42:40 ... to do the same.

Marc Raibert 42:41 Unfortunately though, when you, uh, humans don't really know how they do it.

Lex Fridman 42:44 Yeah.

Marc Raibert 42:45 Right? We, we're-

Lex Fridman 42:46 That's right.

Marc Raibert 42:46 ... coached, we, we have ways of learning, but do we really understand in a physical, in a physics way, uh, what we're doing? Uh, probably most gymnasts and athletes don't know.

Lex Fridman 42:57 So in some way-

Marc Raibert 42:58 (laughs)

Lex Fridman 42:58 ... by building robots, you are in part understanding how humans do, like walking. Most of us walk without considering how we walk, really.

Marc Raibert 43:07 Right.

Lex Fridman 43:08 And how we make it so natural and efficient and all those kinds of things.

Marc Raibert 43:10 Atlas still doesn't walk like a person, and it still doesn't walk quite as gracefully as a person, even though they, it's been getting closer and closer. The running might be close to a human, but the walking is still a, a challenge.

Lex Fridman 43:23 That's interesting, right? Uh, that running is closer to a human. It just shows that the more aggressive and kind of, the more you leap into the unknown, the more natural it is. I mean, walking is kind of falling always, right?

Marc Raibert 43:37 And something weird about the knee, that you can kind of do this folding and unfolding and get it to work out just, a human can get it to work out just right, there's compliances. Compliance meaning springiness, and the-

Lex Fridman 43:49 Yeah, yeah.

Marc Raibert 43:49 ... and the design that are important to-

Lex Fridman 43:51 Yes, yes.

Marc Raibert 43:51 ... how it all works. Well, we used to have a motto at, uh, Boston Dynamics in the early days which was that you have to run before you can walk.

Lex Fridman 43:58 (laughs) Uh, that's a, that's a good motto. 'Cause, uh, you also had Wildcat-

Marc Raibert 44:04 Mm-hmm.

Lex Fridman 44:05 ... which was one of the, uh, along the way towards Spot, which is a quadriped that went 19 miles an hour-

Marc Raibert 44:10 Right.

Lex Fridman 44:10 ... on flat terrain. Is that the fastest you've ever built?

Marc Raibert 44:14 Oh, yeah.

Lex Fridman 44:14 Might be the fastest quadriped in the world. I don't know.

Marc Raibert 44:17 For a quadriped, probably. Of course it was probably the loudest too. So we had this little racing go-kart engine on it. And we would get people from, you know, three buildings away, uh, sending us, you know, complaining about how loud it was.

Lex Fridman 44:31 So at the Leg Lab, I believe most of the robots didn't have knees. (laughs) H- what's the, how do you figure out what is the right number of actuators? What, what are the joints to have? What do you need to have, uh, you know, we humans have knees and all kinds of interesting stuff on the feet. The, the toe is an important part, I guess, for humans. Or maybe it's not. I injured my toe recently, and it made running very unpleasant.

Marc Raibert 44:58 Mm-hmm.

Lex Fridman 44:59 So that seems to be kind of important.

Marc Raibert 45:01 Mm-hmm. Mm-hmm.

Lex Fridman 45:01 So how do you figure out for efficiency, for function, for aesthetics-

Marc Raibert 45:05 Mm-hmm.

Lex Fridman 45:06 ... uh, how many joints to have? How many actuators to have?

Marc Raibert 45:09 Well, there's always a balance between wanting to get where you really wanna get and what's practical to do based on, uh, your resources or what you know and all that. So I mean, the whole idea of the, of the pogo stick was to do a simplification. Obviously it didn't look-

Lex Fridman 45:27 Mm-hmm.

Marc Raibert 45:27 ... like a human. I think, uh, a technical scientist could appreciate that we were capturing some of the things that are important in human locomotion without it looking, uh, like it, without having a knee, an ankle. I'll tell you the first sketch that Ben Brown made, uh, when we were talking about building this thing. It was a very complicated thing with zillions of springs, lots of joints. It looked like, much more like a, uh, a kangaroo or a, or an ostrich or something like that. Things we were paying a lot of attention to at the time. Um, you know, so my job was to, uh, say, "Okay, well, let's do something simpler to get started, and maybe we'll get there at some point."

Lex Fridman 46:10 I just love the idea that you, you two were studying kangaroos and ostriches.

Marc Raibert 46:14 Oh, yeah. We, we did, uh, we filmed and, and digitized, uh, uh, data from horses. I di- I did a dissection of an ostrich at one point, which has absolutely remarkable legs.

Lex Fridman 46:27 Dumb question. Uh, do ostriches have, like, musc- mus- a lot of musculature on the legs or no?

Marc Raibert 46:33 Most of it's up in the feathers, but there's a huge amount going on in the feathers, including a knee joint. The knee joint's way up there.

Lex Fridman 46:40 Mm-hmm.

Marc Raibert 46:40 The thing that's halfway down the leg that looks like a backwards knee is actually the ankle.

Lex Fridman 46:46 Ah.

Marc Raibert 46:46 The thing on the ground which looks like the foot is actually the toes. It's an extended toe.

Lex Fridman 46:51 Fascinating.

Marc Raibert 46:52 But, you know, the basic morphology is in, is the same in, in, uh, all these animals.

Lex Fridman 46:59 What do you think is, uh, the most beautiful movement of an animal? Like, what animal do you think is the coolest-

Marc Raibert 47:05 (laughs)

Lex Fridman 47:06 ... land animal that's cool? 'Cause fish is pretty cool, like the-

Marc Raibert 47:09 Uh-

Lex Fridman 47:09 ... way a fish moves through water, but like-

Marc Raibert 47:10 Yeah.

Lex Fridman 47:10 ... legged locomotion.

Marc Raibert 47:11 You know, the slo-mos of cheetahs running are, are incredible. You know, their, they, there's so much back motion and, uh, you know, grace in their, of course they're moving very fast. Uh, the animals running away from the cheetah are pretty exciting. You know, the pronghorn, uh, which, you know, they, they do this, uh, all four legs at once jump called a prong to kind of confuse the, especially if there's a group of them, to confuse whoever's chasing them.

Lex Fridman 47:39 So they do like a misdirection type of thing?

Marc Raibert 47:41 Yep, they do a misdirection thing. The front on views of the cheetahs running fast where the tail is whipping around to help in the turns, to help stabilize in the turns, those, that's pretty exciting.

Lex Fridman 47:51 'Cause they spend a lot of time in the air, I guess-

Marc Raibert 47:53 Yep.

Lex Fridman 47:53 ... as they're running that fast.

Marc Raibert 47:54 But they also turn very fast.

Lex Fridman 47:57 Is that a tail thing, or is that, do, do you have to have contact with the ground?

Marc Raibert 48:00 Oh, everything in the body is probably helping turn, 'cause they're chasing something that's trying to get away that's also zigzagging around. But I, I, I would be remiss if I didn't say, you know, humans are, are pretty-

Lex Fridman 48:12 (laughs) Right.

Marc Raibert 48:12 ... pretty good too. You know, you watch gymnasts-

Lex Fridman 48:14 Mm-hmm.

Marc Raibert 48:14 ... uh, especially these days, they're doing just incredible, uh, stuff.

Lex Fridman 48:19 Well, like, especially, like, Olympic level gymnasts. See, but there could be, I, there could be cheetahs that are Olympic level. We might be-

Marc Raibert 48:25 (laughs)

Lex Fridman 48:25 ... watching the average cheetah versus, like, there could be a, like, a really special cheetah that can do, like-

Marc Raibert 48:31 You're right.

Lex Fridman 48:32 When did the knees first come into play in, in you building legged robots?

Marc Raibert 48:37 Uh, in BigDog.

Lex Fridman 48:38 BigDog?

Marc Raibert 48:39 Yeah, BigDog came first, and then LittleDog was later.And you know there is a, there's a big compromise there. Uh, human knees have multiple muscles and you could argue that there's, uh... I mean, it's a technical thing about negative work. Uh, when you're, when you're contracting a joint, but you're pushing out, that's negative work.

Lex Fridman 49:03 Mm-hmm.

Marc Raibert 49:03 And if you don't have a place to store that, it can be very expensive to do negative work. And in most of, uh, and in BigDog, there was no place to store negative work, uh, in the knees. But BigDog also had pogo stick, uh, springs down below, so part of the action was to comply in a bouncing motion. You know, later on in Spot, we, we took that out and as we got further and further away from the Leg Lab, uh, we had more, uh, you know, energy-driven controls.

Lex Fridman 49:34 Is there something to be said about, like, knees that go f-

Marc Raibert 49:37 (laughs)

Lex Fridman 49:38 ... uh, forward versus backward?

Marc Raibert 49:40 Sure. There's this idea, uh, called, uh, passive dynamics, which says that although, although you can use computers and actuators to make a motion, a ma- a mechanical system can make a motion just by itself if it gets stimulated the right way. Uh, so, uh, Tad McGear in the, uh, I think in the mid '80s, uh, maybe it was in the late '80s, starting to, started to work on that, and he made this, uh, legged system that could walk down an inclined plane where the legs folded and unfolded and swung forward, you know, doing the, the whole walk- walking motion where the only thi- there was no computer. There were some adjustments to the mechanics so that there were dampers and springs in some places that helped the, uh, the mechanical action happen. It was essentially a mechanical computer. And the idea, the interesting idea there is that it's not all about the brain, uh, telling, dictating to the body what the body should do. The body is a participant in the motion.

Lex Fridman 50:42 So a, a great design for a robot has a mechanical component where the movement is efficient even without a brain?

Marc Raibert 50:49 Yes.

Lex Fridman 50:50 How do you design that?

Marc Raibert 50:52 I think that these days most robots aren't doing that. Most robots are, are basically using the computer to, to govern the motion.

Lex Fridman 51:00 Mm-hmm.

Marc Raibert 51:00 Now the, the brain, though, is taking into account what the mechanical thing can do, uh, and how it's gonna behave. Otherwise, it would have to really forcefully move everything around all the time, which probably some solutions do, but I think you end up with a more efficient and more graceful thing if you're taking into account what the, what the machine wants to

Marc Raibert 51:23 do.

Lex Fridman 51:23 So this might be a good place to mention that you're now, uh, leading up the, the Boston Dynamics AI Institute, newly formed, which is focused more on designing the robots of the future. And I think one of the things, maybe you can tell me the big vision for what's going on, but, uh, one of the things is, uh, this idea that hardware still matters with, with organic design and so on. Maybe before that, can you zoom out and tell me what the vision is for the AI Institute?

Marc Raibert 51:57 You know, I like to talk about intelligence having two parts, an athletic part and a, uh, cognitive part.

Lex Fridman 52:03 Mm-hmm.

Marc Raibert 52:04 And, uh, I think, you know, Boston Dynamics, in my view, has sort of set the standard for, uh, what athletic intelligence can be, and you know, it has to do with all the things we've been talking about. The, the mechanical design, the, the real-time control, the energetics, and that kind of stuff. But obviously, uh, people have another kind of intelligence, and, and animals have another kind of intelligence, you know. We can make a plan. Uh, our meeting started at, at 9:30. I looked up on Google Maps how long it took to walk over here, it was, you know, 20 minutes, so, uh, I decided, okay, I'd leave my house at 9:00, which is what I did.

Lex Fridman 52:41 Mm-hmm.

Marc Raibert 52:42 Um, you know, simple intelligence, but we use that kind of stuff all the time. It's sort of what we think of as going on in our heads. Um, and I think that's in short supply for robots. Most robots are pretty dumb. And as a result, it takes a lot of skilled people to program them to do everything they do, and it takes a long time. And if robots are gonna, you know, satisfy our dreams, uh, they need to be smarter. Uh, so the in- AI Institute is designed to combine that physicality of the athletic side with, uh, the cognitive side. So for instance, we're trying to make robots that can watch a human do a task, uh, understand what it's seeing, and then do the task itself. So sort of OJT for robo- on-the-job training for robots.

Lex Fridman 53:34 Mm-hmm. Mm-hmm.

Marc Raibert 53:35 Uh, as a paradigm. Uh, now, you know, that's pretty hard, uh, and it's, it's sort of science fiction, but our idea is to work on a longer timeframe and, and work on, uh, solving those kinds of problems. And I have a whole list of things that are kind of like in that, in that vein.

Lex Fridman 53:53 Maybe we can just take many of the things you mentioned, just take it as a tangent.

Marc Raibert 53:58 Okay.

Lex Fridman 53:59 Uh, first of all, athletic intelligence is a super cool term. Uh, and that's, uh, that really is intelligence. We humans kind of take it for granted that we're so good at walking and moving about the world.

Marc Raibert 54:10 And using our hands, you know?

Lex Fridman 54:11 And using our hands.

Marc Raibert 54:11 The mechanics of interacting with all, you know, these parts here.

Lex Fridman 54:15 Yeah.

Marc Raibert 54:15 I'll take these two things.

Lex Fridman 54:15 Yeah.

Marc Raibert 54:16 You know? I'm not-

Lex Fridman 54:16 And you've never touched those things before, right?

Marc Raibert 54:17 I'm not loo- I've never tou- Well, I've touched ones like this.

Lex Fridman 54:19 Okay. (laughs)

Marc Raibert 54:20 But look at all the things I can do, right?

Lex Fridman 54:21 Like this.

Marc Raibert 54:21 I can juggle and I'm rotating it-

Lex Fridman 54:23 Yeah.

Marc Raibert 54:23 ... this way. I can rotate it without looking.

Lex Fridman 54:25 Mm-hmm.

Marc Raibert 54:25 I could fetch these things out of my pocket and figure out which one was which-

Lex Fridman 54:29 Mm-hmm.

Marc Raibert 54:29 ... and all that kind of stuff.

Lex Fridman 54:30 Yeah.

Marc Raibert 54:30 And, uh, I don't think we have much of a clue how all that works yet.

Lex Fridman 54:35 Right. And that's, I really like putting that under the banner of athletic, uh, intelligence.

Lex Fridman 54:41 What are the big open problems in athletic intelligence? So Boston Dynamics...... all with Spot, with Atlas, just have shown time and time again, like pushed the limits of what we think is possible with robots. But where do we stand actually if we kinda zoom out? What are the big open problems on the athletic intelligence side?

Marc Raibert 55:01 I mean, one question you could ask that isn't my question but, you know, are they commercially, uh, viable-

Lex Fridman 55:06 Mm-hmm.

Marc Raibert 55:07 ... uh, could, will they increase productivity?

Lex Fridman 55:10 Yeah.

Marc Raibert 55:10 And I think, you know, we're getting very close to that.

Lex Fridman 55:13 Mm-hmm.

Marc Raibert 55:13 Uh, I don't think we're quite there still. You know, most of the robotics companies it's, it's, uh, it's a struggle. You know, it's really the lack of the cognitive side that probably is the biggest barrier at the moment, even for the physically successful robots.

Lex Fridman 55:27 Interesting.

Marc Raibert 55:27 But, uh, you know, your question's a good one. I mean, you can always do a thing that's, uh, more efficient, uh, lighter, more reliable. I'd say reliability. You know, I know that Spot, they've been working very hard, uh, on getting the, the tail of the reliability curve up, and they've made huge progress. So the robots, you know, there, there's, uh, 1,500 of them out there now, uh, many of them being used in, uh, practical applications day in and day out, uh, where, you know, where they have to work reliably.

Lex Fridman 55:59 Mm-hmm.

Marc Raibert 56:00 And, uh, you know, it's very exciting that they've done that. But it takes a huge effort to get that kind of reliability, uh, in the robot. There's cost too. You know, you'd like-

Lex Fridman 56:08 Uh-huh.

Marc Raibert 56:08 ... to get the cost down. Uh, Spots are still pretty expensive, uh, and I don't think that they have to be.

Lex Fridman 56:15 Mm-hmm.

Marc Raibert 56:16 But it takes, you know, a different kind of activity to do that.

Lex Fridman 56:20 Mm-hmm.

Marc Raibert 56:21 Now that, uh, you know, I think, you know, that, uh, Boston Dynamics is owned primarily by, uh, Hyundai now, and I think that the skills of Hyundai in making cars can be brought to bear in, uh, uh, making robots that are less expensive and more reliable and those kinds of things.

Lex Fridman 56:39 So on the cognitive side, uh, for the AI Institute, what's, what's the trade-off between moonshot projects for you and maybe incremental progress?

Marc Raibert 56:50 That's a good question. I think we're, we're using the paradigm called stepping-stones to moonshots.

Lex Fridman 56:55 (laughs)

Marc Raibert 56:56 I do- I don't believe any... That, that was in my original proposal for-

Lex Fridman 57:00 Mm-hmm.

Marc Raibert 57:00 ... the AI Institute, stepping-stones to moonshots. I think if you go more than a year without seeing a tangible status report of where you are, which is the stepping stone, uh, and it could be a simplification, right? You don't necessarily have to solve all the problems of your target goal even though your target goal is gonna take several years. Uh, you know, those, those stepping-stone results give you feedback, uh, give motivation because usually there's some success in there. Uh, and so, you know, that's the mantra, uh, we've been working on. And that's pretty much how, uh, you know, I'd, I'd say Boston Dynamics has worked, uh, you know, where the, you make progress, uh, uh, and show it as you go. Show it to yourself if not to the world.

Lex Fridman 57:45 What does success look like? Like what, what are the, some of the milestones you're, uh, you're chasing?

Marc Raibert 57:52 Well, we've, we've, with watch-understand-do, the project I mentioned before, you know, we've broken that down into, uh, getting some progress with what does meaningfully watching something mean, uh, breaking down, uh, an observation of a person doing something into the components. You know, segment- segmenting, you know, you watch me do something. I'm gonna pick up this thing and put it-

Lex Fridman 58:14 Mm-hmm.

Marc Raibert 58:14 ... down here and stack this on it. Well, it's not obvious if you just look at the raw data, uh, uh, what the sequence of acts are. It's, it's really a creative, intelligent act for you to, to break that down into the pieces and understand them in a way so you could say, "Okay, what skill do I need to accomplish each of those things?" Uh, so we're working on, you know, the front end of, of that kind of a problem where we observe and translate the if- it may be video, it may be live, into, uh, a description of what we think is going on and then trying to map that into skills to accomplish that, and we've been developing skills as well. So, you know, we have kind of multiple stabs at the pieces of, of doing that.

Lex Fridman 58:55 And this is usually video of humans manipulating objects with their hands kind of thing?

Marc Raibert 59:00 Mm-hmm. We're starting out with bicycle repair, some simple-

Lex Fridman 59:03 (laughs)

Marc Raibert 59:03 ... bicycle repair-

Lex Fridman 59:04 Oh, no.

Marc Raibert 59:05 ... tasks.

Lex Fridman 59:05 That seems complicated. That seems-

Marc Raibert 59:06 Well-

Lex Fridman 59:06 ... really complicated.

Marc Raibert 59:07 ... it is. But, but there's some parts of it that aren't, like, uh, putting the seat in, you know, into the... You know, you have a tube that goes inside of another tube-

Lex Fridman 59:16 Sure.

Marc Raibert 59:16 ... and there's a latch. You know, that's, that should be within range.

Lex Fridman 59:19 Is it possible to, to observe, to watch a video like this without having an explicit model of what a bicycle looks like?

Marc Raibert 59:26 I think it is. And I think that's the kind of thing that people don't recognize. Let, let me translate it to navigation.

Lex Fridman 59:33 Mm-hmm.

Marc Raibert 59:33 You know, uh, I think the basic paradigm for navigating a space is to get some kind of sensor that tells you where an obstacle is and what's open, build a map, and then go through the space. But if I, if we were doing on-the-job training where I was giving you a task, I wouldn't have to say anything about the room, right? We came in here-

Lex Fridman 59:52 Mm-hmm.

Marc Raibert 59:53 ... uh, all we did is adjust the chair-

Lex Fridman 59:54 Mm-hmm.

Marc Raibert 59:54 ... but we didn't say anything about the room and, you know, we could navigate it. So I think there's opportunities to build that kind of navigation skill into robots, uh, and we're, you know, we're hoping to be able to do that.

Lex Fridman 1:00:07 So operate successfully under a lot of uncertainty, like...

Marc Raibert 1:00:10 Yeah, and, and lack of specification.

Lex Fridman 1:00:13 Lack of specification.

Marc Raibert 1:00:14 I mean, that's what sort of intelligence is, right? It's kind of dealing with a sit- understanding a situation even though it wasn't explained.

Lex Fridman 1:00:22 So how, how big of a role does machine learning, uh, play in all of this? Is this more and more learning based?

Marc Raibert 1:00:30 You know, since ChatGPT, which is a year ago basically, uh, there's a huge interest in that and a, and a huge, uh, optimism about it. And I think that there's a lot of things that machine learn- that kind of machine learning. Now, of course, there's lots of different kinds of machine learning. I think there's, uh, you know, a lot of interest and optimism about it. I think the, you know, the facts on the ground are that...... doing physical things with physical robots is a little bit different than language.

Lex Fridman 1:01:00 Mm-hmm.

Marc Raibert 1:01:01 And, uh, the tokens, you know, the tokens sort of don't exist. You know, pixel, pixel values aren't like words. Um, but I think that there's a lot that can be done there. We have, uh, uh, we have several people working on machine learning approaches. I don't know if you know, but we, we opened an office in Zurich, uh, recently.

Lex Fridman 1:01:22 Mm-hmm.

Marc Raibert 1:01:22 And, uh, Marco Hutter, who's one of the real leaders in, uh, reinforcement learning for robots, uh, is the, the director of that office. He's still half-time at, uh, ETH, uh, the university there, where he has an unbelievably fantastic lab. And then he's half-time, uh, leading, uh, will be leading oth- efforts in the Zurich office. So we have a healthy, uh, learning component. But there's part of me that still says if you look out in the world at what the most impressive performances are, they're still pretty much, uh ... I hate to use the word traditional, but that's what everybody's calling it, traditional controls, like model predictive control. Uh, you know, the thing, the, the Atlas performances that you've seen are mostly model predictive control. They started to do some learning stuff that's really incredible. I, I don't know if it's all been shown yet, but-

Lex Fridman 1:02:15 Mm-hmm.

Marc Raibert 1:02:15 ... you'll see it over, over time. Um, and then Marco has done some great stuff, and, and others.

Lex Fridman 1:02:21 So especially for the athletic intelligent piece, uh, the traditional approach seems to be the one that still performs the best.

Marc Raibert 1:02:29 I think we're gonna find a ma- a mating of the two, and we'll have the best of both worlds. And we're working on that at the institute

Marc Raibert 1:02:35 too.

Lex Fridman 1:02:36 If I can talk to you about teams. You've built an incredible team at Boston Dynamics before at MIT and CMU, at Boston Dynamics, and now at the AI Institute. A- and you said that there's four components to a great team, uh, technical fearlessness, diligence, intrepidness, and fun, technical fun. Can you explain each? Technical fearlessness, what do you mean by that?

Marc Raibert 1:02:58 Sure. Uh, technical fearlessness means being willing to take on a problem that you don't know how to solve.

Lex Fridman 1:03:03 Mm-hmm.

Marc Raibert 1:03:04 Uh, and, you know, uh, study it, uh, figure out an ac-, an entry point, you know, maybe a simplified version or a simplified solution or something, learn from the stepping stone, and, uh, and go back. And, uh, eventually make a solution that meets your goals. And I think that's really important.

Lex Fridman 1:03:28 The fearlessness comes into play because some of it has never been done before?

Marc Raibert 1:03:32 Yeah (laughs) , and you don't know how to do it, and, you know, there's easier stuff to do in life. Uh, so, you know ... I mean, I don't know, watch, understand, do. (laughs) It's a, it's a mountain of a, of a challenge.

Lex Fridman 1:03:45 So that's the b- really big challenge you're, you're tackling now. Can we watch humans at scale and have robots, by watching humans, become effective actors in the world?

Marc Raibert 1:03:57 Yeah. I mean, we have others like that. I, th- we have one called Inspect-Diagnose-Fix. Like, uh, you know, you, uh, call up the Maytag, uh, repair man. Okay, he's the one who you don't have to call. But you, you, you know, you call up the, the dishwasher repair person, and they come to your house, and they look at your machine. It's already been actually figured out that something doesn't work, but they have to kind of examine it and figure out what, what's wrong and then fix it.

Lex Fridman 1:04:24 Mm-hmm.

Marc Raibert 1:04:25 And, uh, I think robots should be able to do that. Uh, we already, uh, Boston Dynamics already has spot robots collecting data on machines, things like thermal data, reading the gauges, listening to them, getting sounds. And that data are used to determine whether they're healthy or not.

Lex Fridman 1:04:45 Mm-hmm.

Marc Raibert 1:04:46 But the interpretation isn't done by the robots yet. And the, uh, certainly the, the fixing, the diagnosing and the fixing isn't done yet, but I think it could be. And, you know, that's bringing the AI in and combining it with the physical skills to do it.

Lex Fridman 1:05:00 Yeah, and you're referring to the fixing in the physical world. I can't wait until they can fix the-

Marc Raibert 1:05:04 (laughs)

Lex Fridman 1:05:04 ... psychological problems of humans and show up and just talk.

Marc Raibert 1:05:07 Oh. (laughs)

Lex Fridman 1:05:07 Do therapy.

Marc Raibert 1:05:08 No, that's a, that's a different thing.

Lex Fridman 1:05:10 Yeah, it's a different ... Well-

Marc Raibert 1:05:11 But-

Lex Fridman 1:05:11 ... it's all part of the same thing, again, humanity. (laughs) Maybe, maybe. Uh-

Marc Raibert 1:05:17 You mean convincing you it's okay that the dishwasher is broken, just do them for half?

Lex Fridman 1:05:20 Oh, yeah, that's- (laughs)

Marc Raibert 1:05:21 (laughs)

Lex Fridman 1:05:21 Yeah. Yeah, that's very

Narrator 1:05:22 (laughs)

Marc Raibert 1:05:22 ... the marketing approach?

Lex Fridman 1:05:24 Yeah, exactly. It's all, yeah, don't smet- uh, don't, don't sweat the small stuff. (laughs) Yeah, as opposed to fixing the dishwasher-

Marc Raibert 1:05:31 (laughs)

Lex Fridman 1:05:31 ... it'll convince you that it's okay that the dishwasher is broken. It's a different approach. Uh,

Lex Fridman 1:05:37 diligence, why is diligence important?

Marc Raibert 1:05:40 Well, if you want a real robot solution, it can't be, uh, a very narrow solution that's gonna break at the first variation in what the robot does or the environment, if it wasn't exactly as you expected it.

Lex Fridman 1:05:54 Mm-hmm.

Marc Raibert 1:05:55 So how do you get there? I think, uh, having an approach that leaves you unsatisfied until you've embraced the bigger problem is the, is the diligence I'm talking about. And, uh, again, I'll point at Boston Dynamics. I think they've done a ... You know, some of the videos that we had showing the engineer making it hard for the robot to do its task. Uh, Spot opening a door, and then the guy gets there and pushes on the door, so it doesn't open the way it's supposed to, pulling on the, on the rope that's attached to the robot, so its navigation has been screwed up.

Lex Fridman 1:06:29 Mm-hmm.

Marc Raibert 1:06:30 Uh, we have one where the robot's climbing stairs, and the, uh, engineer is tugging on a rope that's pulling it back down the stairs. You know, that's totally different than just the robot seeing the stairs, making a model, putting its feet carefully on each step. But that's what probably robotics needs to succeed. And having that broader, that broader idea that you wanna come up with a robust solution is what I meant by diligence.

Lex Fridman 1:06:54 So really testing it in all conditions, perturbing the system in all kinds of ways.

Marc Raibert 1:06:58 Right.

Lex Fridman 1:06:59 And as a result, creating some epic videos.

Marc Raibert 1:07:01 (laughs)

Lex Fridman 1:07:02 Uh, the legendary-

Marc Raibert 1:07:02 The fun part, the hockey stick.

Lex Fridman 1:07:04 (laughs) And then, yes, tugging on Spot as it's trying to open the door. I mean, the, it's, it's great testing, but it's also...I don't know, it just s- somehow extremely compelling demonstration of robotics in video form.

Marc Raibert 1:07:21 I learned something very early on with the first three-dimensional hopping machine. If you just show a video of it hopping, it's a so what? If you show it falling over a couple of times, and you can see how easily and fast it falls over, then you appreciate what the robot's doing when it's, when it's doing its thing. So I think, you know, you're, the reaction you just gave to the door, the robot getting kind of, uh, interfered with or tested while it's going through the door, it's showing you the scope of the solution.

Lex Fridman 1:07:53 The, the limits of the system, the, the challenges involved in failure. It's th- it's showing both failure and success makes you appreciate the, the success. Yeah. And then, just the way the videos are done in Boston Dynamics, they're incredible 'cause they're not, there's no flash, there's no extra, like, production. It's just raw testing of the robot.

Marc Raibert 1:08:13 Well, you know, I was the final edit for most of the videos up until, uh, until about three years ago or four years ago. And, uh, you know, my theory of the video is no, no explanation. I- if they can't see it, then it's not the right thing. And if you do something worth showing, then let them see it. Don't, don't interfere with, uh, you know, a bunch of titles that slow you down or a bunch of distraction. Just, you know, do something worth showing and then show it.

Lex Fridman 1:08:47 That's brilliant.

Marc Raibert 1:08:48 It's, it's hard, it's hard, though, for, for people to buy into that.

Lex Fridman 1:08:53 Yeah, I mean, people always wanna add more stuff. But the simplicity of just-

Marc Raibert 1:08:58 Yeah.

Lex Fridman 1:08:58 ... do something worth showing and show it, that's brilliant. And don't add extra stuff.

Marc Raibert 1:09:02 And people, people have criticized, uh, especially the, the BigDog videos where there's a human, uh, driving the robot. And, and I understand the criticism now. At the time, we wanted to just show, look, this thing's using its legs to get up the hill, so we focused on showing that, which was, we thought, the, the story.

Lex Fridman 1:09:22 Mm-hmm.

Marc Raibert 1:09:22 The fact that there's a human, so they were thinking about autonomy, whereas we were thinking about the, the mobility.

Lex Fridman 1:09:28 Yeah.

Marc Raibert 1:09:28 Uh, and so, you know, we've, we've adjusted to a lot of things that we see that people care about.

Lex Fridman 1:09:34 Mm-hmm.

Marc Raibert 1:09:35 Uh, trying to be honest. We've always tried to be honest.

Lex Fridman 1:09:38 But also just show cool stuff in its raw form, the limits of the system. The see the system be perturbed and be robust and resilient, and all that kind of stuff. And, uh, and dancing with some music. Uh, intrepidness and fun, so in- intrepid.

Marc Raibert 1:09:57 I mean, it might be the most important ingredient.

Lex Fridman 1:10:00 Sure.

Marc Raibert 1:10:00 And that is, you know, robotics is hard. It's not gonna work right right away, so don't be discouraged is a- all it really means.

Lex Fridman 1:10:08 Mm-hmm.

Marc Raibert 1:10:08 So w- usually when I talk about these things, I show videos, and I show a long string of outtakes, you know. And you know, you have to have, uh, courage to be, to be intrepid when, you know, you work so hard to build your machine, you know, and then you're trying, and it just doesn't do what you thought it would do, what you want it to do. And, uh, you know, you have to stick to it and keep trying.

Lex Fridman 1:10:34 How long ... I mean, we don't often see that, the story behind Spot and Atlas. How long, how many failures was there along the way to get a, you know, a working Atlas, a working Spot, in their early days, even a working BigDog?

Marc Raibert 1:10:49 There's a video of Atlas climbing three big steps, and it's very dynamic, and it's, you know, it's really exciting, a real accomplishment. It took, uh, 109 tries, and we have video of every one of them. You know, we shoot-

Lex Fridman 1:11:02 Mm-hmm.

Marc Raibert 1:11:02 ... everything. Again, we, this is at Boston Dynamics.

Lex Fridman 1:11:06 Mm-hmm.

Marc Raibert 1:11:07 Um, uh, so it took 109 tries. But once it did it, it had a high percentage of success.

Lex Fridman 1:11:13 Mm-hmm.

Marc Raibert 1:11:13 So it's not like we're cheating by just showing the best one. But we do show the evolved, you know, performance, not everything along the way. But the everything along the way is informative, and you know, it shows sort of, uh, there's, you know, stupid things that go wrong.

Lex Fridman 1:11:29 Mm-hmm.

Marc Raibert 1:11:30 Like, like the robot, just when you say go and it collapses right there on the-

Lex Fridman 1:11:34 Mm-hmm.

Marc Raibert 1:11:34 ... start, that doesn't have to do with the steps. Uh, or the perception didn't work right, so you miss the target when you jump, or something breaks, and there's oil flying everywhere. Uh, but that's fun. (laughs)

Lex Fridman 1:11:46 Yeah. So the hardware failures and, and then maybe some software failure.

Marc Raibert 1:11:50 Lots of control of evolution during that time. I think it took six weeks to get that, those, uh, 109 trials. You know, 'cause there was, there was, uh, programming going on, you know. It was, it was actually robot learning, but there were human in the loop, helping with the learning, so all data-driven. But, uh ...

Lex Fridman 1:12:08 Okay. And so, and you always are learning from that failure, so.

Marc Raibert 1:12:12 Right.

Lex Fridman 1:12:14 And (laughs) how do you, how do you protect Atlas from that getting damaged from 109, uh, attempts?

Marc Raibert 1:12:23 It was re- it's remarkable. One of the accomplishments of Atlas is that the engineers have made a machine that's robust enough that it can take that kind of testing where it's falling and stuff and it doesn't break every time. It still breaks, and we had, you know, part of the, the paradigm is to have people to repair stuff. It's, you gotta figure that in if you're gonna do this kind of work. Um, you know, I, I sometimes criticize the people who have their gold-plated thing and they keep it on the shelf, and everybody, and they're afraid to kind of use it. I don't think you can make progress if you're working that way. You kn- you need to be ready to have it break and, and go in there and fix it. It's part of the thing. You know, plan your budget so that you have-

Lex Fridman 1:13:03 (laughs)

Marc Raibert 1:13:03 ... spare parts and a crew and-

Lex Fridman 1:13:06 Yeah.

Marc Raibert 1:13:06 ... all that stuff.

Lex Fridman 1:13:07 Yeah, if it falls 109 (laughs) times, it's okay. Wow. Um, so intrepid, truly. And that applies to Spot, that applies to all the other robots at Boston.

Marc Raibert 1:13:17 Applies to everything. I think it applies to everything anybody tries to do that's worth doing. (laughs)

Lex Fridman 1:13:21 Yeah.... and especially with systems in the real world, right?

Marc Raibert 1:13:24 Yeah.

Lex Fridman 1:13:25 Uh, and so fun.

Marc Raibert 1:13:27 Fun. Technical fun, I usually say.

Lex Fridman 1:13:30 Technical.

Marc Raibert 1:13:30 Have technical fun. I think that, that life as an engineer is really satisfying.

Lex Fridman 1:13:35 Mm-hmm.

Marc Raibert 1:13:36 I think you get to, uh, you know, to some degree, it, it can be like crafts work where you get to do things with your own hands or your own design or whatever your, you know, your media is. And it's very satisfying to be able to just do the work, unlike, you know, a lot of people who have to do something that they don't like doing. I think engineers typically get to do something that they, that they like, and there's a lot of satisfaction from that. Then there's, um, you know, in many cases you can have impact, uh, on the world somehow because you've done something that other people admire. Which is dif- different from the own, just the craft fun of, of building a thing. Uh, so that's a second way that, uh, that being engineer is good. I think the third thing is that the, if you're lucky to be working in a team where you're, uh, getting the benefit of other peoples' skills that are helping you do your thing, uh, you know, none of us has all the skills needed to do, um, most of these projects. And, uh, if you have a team where you're working well with the others, that can be very satisfying. And then if you're an engineer, you also usually get paid.

Lex Fridman 1:14:43 (laughs)

Marc Raibert 1:14:43 And so you-

Lex Fridman 1:14:43 Yeah.

Marc Raibert 1:14:43 ... you kinda get paid four times-

Lex Fridman 1:14:45 Yeah. (laughs)

Marc Raibert 1:14:46 ... uh, in my view of the world.

Lex Fridman 1:14:47 Yeah.

Marc Raibert 1:14:48 So what could be better than that?

Lex Fridman 1:14:49 Get paid to have fun.

Marc Raibert 1:14:50 Yeah.

Lex Fridman 1:14:50 Uh, I mean, what, what do you love about engineering? What, when you say engineering, what does that, what does that mean to you exactly? What is this kinda big thing that we call engineering?

Marc Raibert 1:15:00 I think it's both being a scientist or getting to use science at the same time as being kind of an artist or a creator 'cause you're making some... You know, scientists only get to describe, to, to study what's out there, and engineers get, get to make stuff that didn't exist before.

Lex Fridman 1:15:16 Mm-hmm.

Marc Raibert 1:15:16 And so it's really, I think, a higher calling even though I think most, you know, the public out there thinks science is top and engineering is somehow secondary, but I think it's the other way around.

Lex Fridman 1:15:26 And at the cutting edge, I think when you, when you talk about robotics, there is possibility to do art in that you do like the first of its kind thing. So then there's the ma- production at scale, which is its own beautiful thing, but when you do the first new robot or the first new thing, that's a possibility to create something totally new that is an art-

Marc Raibert 1:15:48 I mean, br- bringing metal to life or a machine to life is kind of, is fun, and, uh, you know, it was fun doing the da- the dancing videos where, uh, got a huge, you know, public response, and we're gonna do more. We're gonna do some at, we're doing some at the institute, and we'll, we'll do more.

Lex Fridman 1:16:05 Well, that metal to life moment, I mean, to me, that's still magical. Like, uh, when, when, um, inanimate object comes to life, that's still, like to me, it's to this day still an incredible moment that human intelligence can create systems that instill life or whatever that is in- into an inanimate objects. It's, it's, it's really tr- it's truly magical, especially when it's at the scale of, that humans can perceive and appreciate like directly.

Marc Raibert 1:16:37 But I think sort of with going, going back to the pieces of that, you know, you design a linkage that turns out to be half the weight and just as strong. That's, that's very satisfying.

Lex Fridman 1:16:48 That's very satisfying, yeah.

Marc Raibert 1:16:48 And, you know, there are people who do that, and it's, it's a creative, a creative act.

Lex Fridman 1:16:53 Uh, what, what to you is the most beautiful about robotics?

Marc Raibert 1:16:57 (laughs)

Lex Fridman 1:16:59 Sorry for the big romantic question.

Marc Raibert 1:17:01 I think having the robots move in a way that's, uh, uh, evocative of life is, is pretty exciting.

Lex Fridman 1:17:08 So the elegance of movement?

Marc Raibert 1:17:09 Yeah. Or, or if it's a high performance act where it's doing it, you know, faster, bigger than, uh, than other robots. Usually we're not doing it bigger, faster than people, but, you know, we're getting there in a few narrow dimensions.

Lex Fridman 1:17:22 So faster, bigger, smoother, more elegant, more graceful?

Marc Raibert 1:17:27 Yeah. I mean, I'd like to do dancing that, that starts... You know, we're nowhere near the, uh, the dancing capabilities of a human. We h- we've been having a ballerina in who's, uh, kind of a well-known ballerina, and she's been programming, uh, the robot. We've been working on the tools that can make it so that she can use her way of talking, uh, you know, way of doing a choreography or something like that more accessible, uh, to, uh, to get the robot to do things. And she's starting to produce some interesting stuff.

Lex Fridman 1:17:58 Well, we should mention that there is a choreography tool.

Marc Raibert 1:18:00 There is.

Lex Fridman 1:18:01 (laughs) I, I mean, I, I, I guess I saw versions of it, uh, (laughs) which is pretty cool. You can kinda-

Marc Raibert 1:18:08 (laughs)

Lex Fridman 1:18:08 ... at, at slices of time-

Marc Raibert 1:18:10 Uh-huh.

Lex Fridman 1:18:10 ... control different parts at the high level, the movement of the robot and Spot and the others.

Marc Raibert 1:18:14 We hope to take that forward and make it, you know, more tuned to how, uh, the dance world wants to talk, wants to communicate, and, and get better performances. I mean, we've do- done a lot, but there's still a lot possible. And I'd like to have, uh, performances where the robots are dancing with people. So right now almost everything that we've done on dancing, uh, is to a fixed time base. So once you press go, the robot does its thing and plays out its thing. It's not listening, it's not watching-

Lex Fridman 1:18:46 Mm-hmm.

Marc Raibert 1:18:46 ... but I think it should do those things.

Lex Fridman 1:18:49 I think I would love to see a professional ballerina like alone in a room with a robot slowly teaching the robot.

Marc Raibert 1:18:56 (laughs)

Lex Fridman 1:18:56 Just actually the, the process of a clueless robot trying to figure out a, a small little piece of a dance. So it's not like... Uh, 'cause right now Atlas and Spot have done like perfect dancing-

Marc Raibert 1:19:08 Right.

Lex Fridman 1:19:09 ... uh, to a beat and so on. Well, and so, you know, uh, to a degree, but like, uh, the learning process of interacting with a human would be like incredible to watch.

Marc Raibert 1:19:19 One of the cool things going on, y- you know that there's a class at Brown