Lex Fridman PodcastBoris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics | Lex Fridman Podcast #241
EVERY SPOKEN WORD
150 min read · 30,048 words- 0:00 – 1:08
Introduction
- LFLex Fridman
The following is a conversation with Boris Sofman, who is the senior director of engineering and head of trucking at Waymo, the autonomous vehicle company, formerly the Google self-driving car project. Before that, Boris was the co-founder and CEO of Anki, a robotics company that created Cozmo, which, in my opinion, is one of the most incredible social robots ever built. It's a toy robot, but one with an emotional intelligence that creates a fun and engaging human-robot interaction. It was truly sad for me to see Anki shut down when it did. I had high hopes for those little robots. We talk about this story and the future of autonomous trucks, vehicles, and robotics in general. I spoke with Steve Viscelli recently on episode 237 about the human side of trucking. This episode looks more at the robotic side. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description, and now here's my conversation with Boris Sofman.
- 1:08 – 6:49
Robots in science fiction
- LFLex Fridman
Who is your favorite robot in science fiction, books or movies?
- BSBoris Sofman
WALL-E and R2-D2, where they were able to convey such an incredible degree of intent, emotion, and kinda character attachment without having any language whatsoever, um, and just purely through the emo- richness of em- emotional interaction, so those are fantastic. And then, uh, the Terminator series is just, like, really (laughs) in front of my eye-
- LFLex Fridman
(laughs) Okay.
- BSBoris Sofman
... pretty wide-
- LFLex Fridman
Jumped from WALL- (laughs)
- BSBoris Sofman
... wide range, right? Uh, but, uh, I kinda love this, uh, dynamic where you have this, like, incredible Terminator itself that, that Arnold played, but, uh, and then he was kind of like the inferior, like, previous generation version that was, like, totally outmatched, uh, you know, in terms of kinda specs by the new one, but, you know, still kind of, like, held his own. And so it was kind of interesting where you, you realize how many, how many levels there are on the spectrum from human to kind of potentials in AI and robotics to, uh, futures. And so, yeah, that movie really, uh, as much as it was, like, kind of a dark world in a way, was actually quite fascinating, gets the imagination going.
- LFLex Fridman
Well, from an engineering perspective, both the movies you mentioned, uh, WALL-E and, um, Terminator, uh, the first one is probably achievable, you know, humanoid robot, maybe not with, like, the realism in terms of skin and so on, but that humanoid form, we have that humanoid form. It seems like a compelling form. Maybe the challenge is it's super expensive to engine- to, to build, but you can imagine maybe not a machine of war-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... but you can imagine Terminator-type robots walking around.
- BSBoris Sofman
Yeah.
- LFLex Fridman
Uh, and then the same obviously w- with WALL-E. You've basically... So for people who don't know, you, uh, created the company Anki that created, uh, a small robot with a big personality called Cozmo that just... It does exactly what WALL-E does, which is somehow with very few basic visual tools is able to communicate a depth of emotion, and that's fascinating. Uh, but then again, the humanoid form is, uh, super compelling. So, like, C- uh, Cozmo is very distant from a humanoid form-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... and then the Terminator has a humanoid form, and you can imagine both of those actually being in our society.
- BSBoris Sofman
It's true, and it's interesting because, um, it- it was very intentional to go really far away from human form when you think about a character like Cozmo or like WALL-E, where, um, you can completely rethink, uh, the constraints you put on that character, um, what tools you leverage, and then how you actually create a personality, uh, and a level of intelligence interactivity that actually matches the constraints that you're under, whether it's, uh, mechanical or sensors or AI of the day. This is why I almost- um, was always very surprised by how much energy people put towards trying to replicate human form-
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
... in a robot because you actually take on some pretty significant, um, kinda constraints and, and downsides when you do that.
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
Um, the first of which is obviously the cost, where it just- the, the articulation of a human body is just so, like, magical, um, in both the precision as well as the dimensionality, that to replicate that even in its clo- reasonably close form takes, like, a giant amount of joints and actuators and, uh, emotion and, and, you know, sensors and encoders and so forth. But then, um, you're almost, like, setting an expectation that the closer you try to get to human form, the more you expect the strengths to match, and that's not the way AI works is there's places where you're way stronger, and there's places where you're weaker. And by moving away from human form, you can actually, uh, change the rules and embrace your strengths and bypass your weaknesses.
- LFLex Fridman
And at the same time, the human form, like, has way too many degrees of freedom to play with. It's, it's kind of con- counterintuitive just as you're saying, but when you have fewer constraints, it's almost harder to master the, the communication of emotion. Like, you see this with cartoons, like stick figures. You can communicate quite a lot with just very minimal, like two dots for eyes and a line for, for a smile. I think it- like, you can almost communicate arbitrary levels of emotion with just two dots and a line.
- BSBoris Sofman
Yeah.
- LFLex Fridman
And, like, that's enough, and if you focus on just that, you can communicate the full range, and then-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... you, like, if you do that, then you can focus on the actual magic of, of, uh, human and dot-line interaction versus all the engineering mess.
- BSBoris Sofman
That's right, like, dimensionality, voice, all these sort of things, they actually become a crutch-
- LFLex Fridman
Yeah.
- BSBoris Sofman
... where you get lost in a search space almost. Um, and so some of the best animators that we've worked with, um, they almost, like, study when they come up, uh, you know, kind of in buil- in building their expertise by forcing these, um, projects where all you have is, like, a ball that can, like, kind of jump and manipulate itself or, like, really, really, like-... aggressive constraints where you're forced to kind of ex- ex- extract the deepest level of emotion. And so, in a lot of ways, um, you know, when we tho- when we thought about Cozmo, it was like, "You're, you're right." Like, our... If we had to, like, describe it in, like, one small phrase, it was bringing a Pixar character to life in the real world. It's, uh, it's- it's what we were going for. And, um, in a lot of ways what was interesting is that with, like, WALL-E, which we studied incredibly deeply, and in fact some of our team were, you know, kind of, uh, had worked previously at, um, at Pixar and on that project. Um, they intentionally constrained WALL-E as well, even though in an animated film you could do whatever you wanted to because it forced you to, like, really saturate the smaller amount of dimensions. But, uh, you sometimes end up getting a far more beautiful output, um, because you're pushing at the extremes, um, o- of this emotional space in a way that you just wouldn't because you get lost in a surface area, uh, if you have, like, something that is just infinitely articulable.
- 6:49 – 32:04
Cozmo
- BSBoris Sofman
- LFLex Fridman
So if we backtrack a little bit, and, uh, you thought of Cozmo in 2011, in 2013 actually, uh, designed and built it. What is Anki? What is Cozmo? I guess, who is Cozmo? (laughs)
- BSBoris Sofman
Who is Cozmo, yeah.
- LFLex Fridman
And, uh, what was the vision behind this incredible little robot?
- BSBoris Sofman
We started, uh, Anki back in, like, like, while we were still in graduate school. So myself and my two co-founders, we were PhD students, uh, in the Robotics Institute at Carnegie Mellon. Um, and so we were, uh, studying robotics, AI, machine learning, kind of diff- you know, different, uh, uh, areas. One of my co-founders was working on walking robots, uh, you know, uh, for a period of time. And so we all had a, um, a bit of a really deep, kind of a deeper passion for applications of robotics and AI, where, um, th- there's like a spectrum where there's people that get, like, really fascinated by the theory of AI and machine learning and robotics where, um, whether it gets applied in the near future or not is less of a kind of factor on them, but they love the pursuit of the, like, the challenge. And that's necessary, and there's a lot of incredible breakthroughs that happen there. We're probably closer to the other end of the spectrum where we love the technology and the, um, and all the evolution of it, but we were really driven by applications. Like, how can you really reinvent experiences and functionality and build value that wouldn't have been possible without, um, these approaches? And- and that's what drove us, and we had a, kind of some experiences through previous jobs and internships where we, like, got to see the applied side of robotics. And at that time, there was actually relatively few applications of robotics, um, that were outside of, um, you know, pure research or industrial applications, um, military applications and so forth. There were very few outside of it. So maybe, you know, iRobot was, like, one exception and then maybe there were a few others, but for the most part there weren't that many. And so we got excited about consumer applications of robotics where you could leverage w- way higher levels of intelligence, um, through software to create value and experiences that were just not possible, um, in- i- in those fields today. Um, and we saw kind of a- a- a pretty wide range of applications, um, that varied in the complexity of what it would take to actually solve those. And what we wanted to do was to commercialize this into a company, but actually do a bottoms-up approach where we could have a huge impact in a space that was ripe to have an impact at that time, and then build up off of that and move into other areas. And entertainment became the place to start-
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
... because, um, you had relatively little innovation in the toy space, uh, and entertainment space. You had these really rich, um, experiences in video games and, uh, and movies, but there was, like, this chasm in between. And so we thought that we could really reinvent that experience and there was a- a really fascinating transition technically that was happening at the time where the cost of components was plummeting because of the mobile phone industry and then the smartphone industry. And so the cost of a microcontroller, of a camera, of a motor, of memory, of microphones, cameras was dropping by orders of magnitude.
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
And then on top of that with the iPhone coming out in 2000, uh, I think it was 2007 I believe, uh-
- LFLex Fridman
Mm-hmm. Yeah, this is right.
- BSBoris Sofman
Um, you s- it- it started to become apparent within a couple of years that this could become a really incredible interface device and the brain with much more computation behind a physical world experience-
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
... that wouldn't have been possible previously. Um, and so, um, we really got excited about that and how we pushed all the complexity from the physical world into software by using really inexpensive components, but putting huge amounts of complexity into the AI side. And so Cozmo became our second product, and then the one that we're probably most proud of. The idea there was to create a physical character that had enough understanding and awareness of the physical world around it and the context that mattered to feel like, uh, like he was alive. Um, and, uh, to be able to have these, like, emotional kind of connections and experiences with people that you would typically only find, uh, inside of a movie. And the motivation very much was- was Pixar, like, we had an incredible, uh, respect and appreciation for what they were able to, um, build in this, like, really beautiful fashion in film. Um, but it was always like a, you know, one, it was virtual, and two, it was like a story on rails that had no interactivity to it. It was, uh, very fixed, and it obviously had a magic to it, but where you really start to hit like a different level of experience is when you're actually able to physically interact with our robot.
- LFLex Fridman
Yeah. And then that was your idea with Anki, like, the first product was the cars.
- BSBoris Sofman
Yeah.
- LFLex Fridman
So basically you take, you take a toy, you add intelligence into it in the same way you would add intelligence into AI systems within a video game, but you're now bringing it into the physical space. So the idea is- is really brilliant, which is you're basically bringing video games to life.
- BSBoris Sofman
Exactly. That's exactly right. We literally use that exact same phrase because in the case of DRIVE, this was a parallel of the racing genre.
- LFLex Fridman
Yeah.
- BSBoris Sofman
And the goal was to effectively have a physical racing experience, but have a virtual state at all times that matches what's happening in the physical world. And then you can have a video game off of that, and you can have, uh, different characters, different traits for your, uh, the cars, um, weapons and interactions and special abilities and all these sort of things that you think of virtually, but then you can have it physically. And, um, one of the things that we were, like, really surprised by that really stood out and immediately led us to really, like, kind of accelerate the path towards, um, Cozmo is that things that feel like they're really constrained and simple in the physical world, they have an amplified impact on people, where the exact same experience virtually would not have anywhere near the impact, but seeing it physically really stood out. And so effectively we've, with- with DRIVE we were creating a video game engine for the physical world, um, and then with Cozmo we expanded that video game engine to create a character and, uh-... uh, kind of an animation and interaction engine on top of it that allowed us to start to create these much more rich experiences. And a lot of those elements were, uh, almost like a proving ground for what would human/robot interaction feel like in a domain that's much more forgiving, where you can make mistakes in a game.
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
It's okay if like, uh, if, you know, c- car goes off the track or if a, if Cozmo makes a mistake. Um, and what's funny is actually we were so worried about that, um, in reality we realized very quickly that those mistakes can be endearing and if you make a mistake, as long as you realize you made a mistake and have the right emotional reaction to it, it builds even more empathy-
- LFLex Fridman
Yeah.
- BSBoris Sofman
... with the character.
- LFLex Fridman
Yeah, that's brilliant.
- BSBoris Sofman
Um, which is-
- LFLex Fridman
Exactly. So when, uh, the, the thing you're optimizing for is fun, you have so much more freedom to fail, to explore. And, and also in the toys space, like all of this is really brilliant and I, I gotta ask you, backtrack. It seems for a roboticist to take a s- jump in, into the direction of fun is a brilliant move because one, you have the freedom to explore, to design, all those kinds of things, and you can also build cheap robots.
- BSBoris Sofman
Yeah.
- LFLex Fridman
Like you don't have to... Like if, if you're not chasing perfection and like toys, it's understood that you can go cheaper.
- BSBoris Sofman
Yeah.
- LFLex Fridman
Which means in robot it's still expensive but it's actually affordable by a large number of people. So it's a really brilliant space to explore.
- 32:04 – 38:59
AI companions
- BSBoris Sofman
future experiences.
- LFLex Fridman
So, if you look out into the future, do you think we will have beyond a particular game, you know, a companion, like, uh, like Her, like the movie Her or, like, a Kosmo that's kinda asks you how your day went too, right?
- BSBoris Sofman
Yeah.
- LFLex Fridman
Ma- You know, like a friend.
- BSBoris Sofman
Yeah.
- LFLex Fridman
Do, would you... How, how many years away from that do you think we are? What's your intuition?
- BSBoris Sofman
Good question. So, I think the idea of a different type of character, like more closer to, like, kind of a pet style companionship-
- LFLex Fridman
Yes, that's-
- BSBoris Sofman
... will come way faster. Um, and there's a few reasons. One is, like, to, to do something like in Her, that's, like, all, effectively almost general AI, and the bar is so high that if you miss it by a bit, you hit the uncanny valley where it just becomes creepy and, like, and not, um, not, not appealing, um, because the closer you try to get to a human in form and interface and voice, the harder it becomes. Whereas, you have way more flexibility on still landing a really great experience if you embrace the idea of a character. And that's why, um, one of the other reasons why we didn't have a voice, uh, and also why, like, a lot of video game characters, uh, like, Sims for example, does not have a voice when you, uh, when you think about it. It was, uh, it wasn't just a cost savings, like, for them. It was actually, for all of these purposes, it was because when you have a voice you immediately narrow down the appeal to some particular demographic or age range or, um, kind of style or gender. Uh, if you don't have a voice, people interpret what they want to interpret, um, and an eight year old might get a very different interpretation than a 40 year old, um, but you create a dynamic range. And so you just, you can lean into these advantages much more, um, in something that doesn't resemble a human. And so that'll come faster.
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
Uh, I don't know when a human-like... That's just, uh, still, like, not... just complete R&D at this point. The, the chat interfaces are getting way more interesting and, and richer, but it's still a long way to go to kind of pass the test of, you know.
- LFLex Fridman
Well, let me, like, let's consider, like, let me play devil's advocate. So Google is a very large company that's servicing a l- uh, it's creating a very com- compelling product that wants to provide a service to a lot of people. But let's go outside of that. Th- you said characters.
- BSBoris Sofman
Yeah.
- LFLex Fridman
It feels like... A- and you also said that it requires general intelligence to be a successful participant in a relationship, which could explain why I'm single. This, this very...
- BSBoris Sofman
(laughs)
- LFLex Fridman
But the... I, I honestly wanna push back on that a little bit could... because I feel like is it possible that if you're just good at playing a character-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... you know, in, in a movie there's a bunch of characters. If you just understand what creates compelling characters and then you, you just are that character and you exist in the world and other people find you and they connect with you just like you do-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... when you talk to somebody at a bar, "I like this character. This character's kind of shady, I don't like them," you pick the ones that you like and, you know, maybe it's somebody that's, uh, reminds you of your fa- father or mother, I don't know what it is, but the, the, the Freudian thing, but th- there's some kind of connection that happens and that's, that, that's the Kosmo you connect to.
- BSBoris Sofman
Yeah.
- LFLex Fridman
That's the future Kosmo you connect... And that's... So, so I guess the statement I'm trying to make, is it possible to achieve a depth of friendship without solving general intelligence?
- BSBoris Sofman
Yeah. I think so. And it's about intelligent kind of constraints, right?
- LFLex Fridman
Right.
- BSBoris Sofman
And just, uh, you set expectations and constraints such that in the space that's left you can be successful. And so, you can do that by having a very focused domain that you can operate in. For example, you're a customer support agent for a particular product-
- LFLex Fridman
Right.
- BSBoris Sofman
... and you create intelligence and a good interface around that. Or, uh, you know, kind of in the personal companionship side, you can't be everything to, uh, ac- across the board. You, you kind of solve those constraints. And I think, uh, I think it's possible. My, my worry is I, like I... Right now, I don't see anybody that has picked up on where kind of Kosmo left off-
- LFLex Fridman
Yes.
- BSBoris Sofman
... and is pushing on it in the same way. And so, I don't know if it's a sort of thing where similar to, like, how, you know, in dot-com there were all these concepts that we considered, like, you know, that didn't work out or, like, failed or, like, were too early or whatnot and then 20 years later you have these, like, incredible successes on almost the same concept. Like, it might be that sort of thing where, like, there's another pass at it that happens in 5 years or in 10 years. But, um, it does feel like that appreciation of that, like, the, this, the three-legged stool, if you will, between, like, you know, the hardware, the AI and the character. Um, that balance, it's hard to, uh, I'm not aware of, of, uh, any pro- anywhere right now where, like, that same kind of aggressive drive with the value on the character is, uh, is happening. And so...
- LFLex Fridman
To me-... just a prediction, exactly as you said, something that looks awfully a lot like Cozmo, not in the actual physical form, but in the three-legged stool. Something like that, in some number of years, will be a trillion-dollar company. I don't understand, like it's obvious to me-
- 38:59 – 1:04:33
Anki
- BSBoris Sofman
- LFLex Fridman
So in 2019, uh, Anki, the company that created Cozmo, the company that you started, had to shut down. How did you feel at that time?
- BSBoris Sofman
Yeah. It was tough. Uh, that was a really emotional stretch and it was pr- really tough year one, like about a year ah- ahead of that was actually a pretty brutal stretch because we were, um, kind of life- life or death on many, many moments, um, just navigating these insane kind of just ups and downs and, um, barriers. And the thing that made it, like, um, like just- just sort of rewinding a tiny bit, like what, you know, what ended up being really challenging about it as a business where is, um, from a commercial standpoint and customer reception standpoint, there's a lot of things you could point to that were like, you know, pretty big successes. Sold millions of units, uh, like, you know, got to like pretty serious revenue, like ki- kind of close to 100 million annual revenue, um, uh, number one kind of product in kind of various categories. But it was pretty expensive, ended up being very seasonal where something like 85% of our volume was in Q4, um, because it was a p- you know, a present and- and it was expensive to market it and explain it and so forth. Um, and even though- though the volume was like really sizable and like the reviews were really fantastic, um, forecasting and planning for it and managing the cash operations was just brutal. Like, it was absolutely brutal. You don't think about this when you're starting a company or when you have a few million in, you know, in- in revenue because it's just your biggest costs are kind of just your headcount and operations and everything's ahead of you. But we got to a point where, um, you know, you, if you look at the entire year, you have to operate your company, pay all, you know, the people and so forth, you have to pay for the manufacturing, the marketing and everything else to do your sales in mostly November, December, and then get paid in December, January by retailers. And those swings were pretty, um, were really rough, um, and just made it like so difficult. Because the more successful you became, the more wild those swings became, um, because you'd have to like spend, you know, tens of millions of dollars on inventory, tens of millions of dollars on marketing, and tens of millions of dollars on payroll and everything else. And then-
- LFLex Fridman
Just the bigger dip and then you waiting for the Q4.
- BSBoris Sofman
Wild. Yeah, and it's not a business that like is recurring kind of month to month and predictable and it's just... And then you're locking in your forecasts in July, um, you know, maybe August, uh, if you're lucky. Um, and, uh, and it's also like very hit driven and seasonal where like you don't have the sort of continued, uh, kind of slow growth like you do in some other, uh, consumer electronics industries. And so before then, like hardware kind of like went out of favor too. And so you had Fitbit and GoPro drop from 10 billion revenue to 1 billion revenue and hardware companies are getting valued at like 1X revenue oftentimes-
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
... um, which is tough, right? And so we effectively kind of got caught in the middle where we were trying to quickly evolve out of entertainment and move into some other categories, but you can't let go of that business because like that's what you're valued on, that's what you're raising money on. Um, but there was no path to pro- kind of pure profitability just there because it was, you know, such, you know, uh, specific type of price points and so forth. And so, um, we tried really hard to make that transition and, um, yeah, we had a- a financing round that fell apart at the last second and effectively there was just no path to kind of get through that and get to the next kind of like holiday season. And so we ended up, um, uh, selling some of the assets and kind of winding down the company. It was, uh, it was brutal. Like we... I was very transparent with the company, like, and the- the team while we were going through it, where actually despite how challenging that period was, very few people left. I mean, like people loved the vision, the team, the culture, the like kind of chemistry and kind of what we were doing. There was just a huge amount of pride there and that we wanted to see it through and we felt like we had a shot to kind of get through these checkpoints. Um, we ended up, uh... And I mean, by brutal, I mean like literally like days of cash, like three, four different times, uh, runway like in the year, you know, kind of before it, um, where you're like playing games of chicken on negotiating credit line-... timelines and, like, repayment, uh, terms and how to get, like, a bridge loan from an investor. It's just like-
- LFLex Fridman
Yeah.
- BSBoris Sofman
... level of stress that, like, as, as hard as things might be anywhere else, like, you'll never come w- you'll come close to that where you feel that, like, responsibility for, you know, 200 plus people, right? Um, and so we were very transparent during our fundraise on who we're talking to, the challenges, um, that we have, w- how it's going and when things were going well, when things were tough. Um, and so it wasn't a complete shock when it happened, but it was just very emotional where, like, I, you know, like, you know, when we announced it finally that, like, um, you know, we've... you know, basically we're just, like, watching kind of, like, you know, the runway and trying to kind of time it, and when we realized that, like, we didn't have any more outs, we wanted to, like, kind of wind it down, make sure that it was, like, clean and, you know, we could, like, kind of take care of people the best we could. But yeah, like, broke down crying at all, you know, hands and somebody e- else had to step in for a bit. And like, it was just very, very emotional, but the beautiful part is, like, afterwards, like, everybody stayed at the office till, like, 2:00, 3:00 in the morning just, like, drinking and hanging out and telling stories and celebrating, and it was just, like, one of the best, uh... for many people was, like, the best kind of work experience that they had, and there was a lot of pride in what we did. And there wasn't anything obviously we could point to that like, "Hey, if only we had done that different, things would have been completely different." It was just, like, the physics didn't line up, uh, and, uh, um, but the experience was pretty, uh, incredible, but it was hard. Like, it was, uh... it had this feeling that there was just, like, incredible beauty in both the technology and products and the team that, um, uh, you know, there's, there's a lot there that, like, in the, you know, right context could have been, uh, pretty incredible, but it was, um, emotional, just...
- LFLex Fridman
(sighs) Yeah, just thinking, I mean, just looking at this company, like you said, the product and technology, but the vision, the implementation, you got the cost down very low.
- BSBoris Sofman
Yeah.
- LFLex Fridman
And the p- the compelling, the nature of the product was great. So many robotics companies failed at this, at they- o- the robot was too expensive, it didn't have the personality, it didn't really provide any value, like, a sufficient value to justify the price. So, like, you, you succeeded where basically every single other robotics company, or most of them that are, like, go in the category of social robotics, have kind of failed. And, I mean, it, it's, um, it's, it's quite tragic. I remember, uh, reading that, and I'm not sure if I talked to you before that happened or not, but I remember, you know, I'm distant from this. I remember being heartbroken reading that because, like, if, if Cozmo's not gonna succeed, what is going to succeed? 'Cause that to me was incredible. Like, the, the, the- it wo- it was an incredible idea. Cost is down the minim- the, the, the, um... it's just, like, the most minimal design in physical form that you could do. It's really compelling. The balance of games. So it's, it's a, it's a fun toy, it's a great gift for all kinds of age groups, right?
- BSBoris Sofman
Yeah.
- LFLex Fridman
It's just, it's, it's compelling in every single way and it seemed like, uh, it was a, a huge success and it, it, it failing was... I don't know, there was heartbreak on many levels for me just as an external observer, is I was thinking, "How hard is it to run a business?"
- BSBoris Sofman
(laughs)
- LFLex Fridman
That's, that's what I was thinking-
- BSBoris Sofman
Oh.
- LFLex Fridman
... like if this failed, this must have failed because the... it's obviously not, like, yeah, it's bu- it's business.
- BSBoris Sofman
Yeah.
- LFLex Fridman
May- maybe it's some aspect of the manufacturing and so on.
- BSBoris Sofman
Yeah.
- LFLex Fridman
But I'm now realizing it's also not just that, it's-
- BSBoris Sofman
Yeah, and then, and-
- LFLex Fridman
... sales, marketing, all those-
- BSBoris Sofman
Oh, it's everything, right? Like, how do you explain something that's like a new category to people-
- LFLex Fridman
Yeah.
- BSBoris Sofman
... that, like, have all these predispositions? And so, like, uh, you know, it l- it, it had some of the hardest elements of... if you were to pick a business, it had the, some of the hardest, uh, um, customer dynamics because, like, to sell a $150 product, you got to convince both the child who want it and the parents to agree that it's valuable. So you're having, like, this dual-prong marketing challenge. You have manufacturing, you have, like, really high precision on the components that you need, you have the AI challenges. So, there were a lot of tough elements, but it was this feeling where, like, it was just really great alignment of unique strength across kind of, like, all these different areas. Just like incredible, like, you know, kind of character and animation team between this, like, Carlos, and there was, like, a character director, Day, that came on board and, like, you know, really great people there. The AI side, the, um, uh, the manufacturing, the, you know, where, um, like, never missing a launch, right? And Ashley, you know, he kind of hidden that quality. It was, um, yeah, it was, it was heartbreaking, but, uh, here's one neat thing is like we had, we had so much, like, fan mail from kind of kids with their parent- parents, like...
- LFLex Fridman
(laughs)
- BSBoris Sofman
I actually, like, there was a bunch that collected in the end-
- LFLex Fridman
Yeah.
- 1:04:33 – 1:36:10
Waymo Via
- LFLex Fridman
you happen to be one of the greatest, at this point, roboticists ever, because you created, uh, this little guy. You were part, obviously, of a great team that created the, the little guy with a deep personality, and are now switching to an entirely, well, maybe not entirely, but a different fascinating, impactful robotics problem, which is autonomous driving, and more specifically the biggest version of autonomous driving, which is autonomous trucking. So you are at Waymo now. Can you give us a big picture overview? What is Waymo? What is Waymo Driver? What is Waymo One?
- BSBoris Sofman
Yeah.
- LFLex Fridman
What is Waymo Via? Can you give an overview of the company and the vision behind the company?
- BSBoris Sofman
For sure. Waymo, b- by the way, is just, has been eye-opening on just how incredible the, uh, the people and the talent is and how in one company you almost have to create, I don't know, 30 companies worth of like-
- LFLex Fridman
(laughs)
- BSBoris Sofman
... technology and capability to like kind of solve the full spectrum of it. So, um, yeah, so I've, I've been at Waymo since, um, 2019, so about two and a half years. So Waymo is, uh, focused on building what we call a driver, which is, uh, creating the ability to have autonomous driving across different environments, vehicle platforms, domains and use cases. Uh, you know, as you know, it got started in, uh, uh, 2009. It was a lo- uh, almost like an immediate successor to the grand challenge in urban challenges that were like incredible, uh, kind of catalysts for this whole space. Um, and so Google started this project, and then eventually Waymo spun out. And so what Waymo's doing is creating, uh, the systems, both, you know, hardware, software, infrastructure, and everything that goes into it to enable and to commercialize autonomous driving. This hits on consumer transportation and ride sharing and kind of vehicles in urban environments. Um, and as you mentioned, it hits on autonomous trucking to, uh, to transport, um, goods. So in a lot of ways, it's transporting people and transporting goods. Um, but at the end of the day, the underlying capabilities that are required to do that are surprisingly better aligned than one might expect, um, where it's the fundamentals of, um, of being able to understand the world around you, process it, make intelligent decisions, and prove that we are at a, a level of safety that enables, uh, large scale autonomy.
- LFLex Fridman
So from a branding perspective, sort of, uh, Waymo Driver is the system that's irrespective of a particular, uh, v- vehicle it's operating-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... and there's... You have a set of sensors that perceive the world, can act in that world, and move this, whatever the vehicle is through the wo-
- BSBoris Sofman
Pla- yeah, vehicle platform. That's right. And so in the same way that you have a driver's license and like your ability-
- LFLex Fridman
(laughs) Yeah.
- BSBoris Sofman
... to drive isn't tied to a particular make and model of a car, and of course there's special licenses for other types-
- LFLex Fridman
Yeah.
- BSBoris Sofman
... of vehicles, but the fundamentals of a, of a human driver, very, very largely carry over. And then there's uniquenesses related to a particular environment or domain or a particular, um, vehicle type that kind of add some extra additive challenges. But that's exactly right. It's the underlying systems that enable a phy- a, a physical vehicle without a human driver to, uh, very successfully accomplish the task that previously, um, wa- wasn't possible, um-
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
... without, um, you know, hu- 100% human driving.
- LFLex Fridman
Mm-hmm. And then there's Waymo One, which is the transporting people-
- BSBoris Sofman
That's right.
- LFLex Fridman
... from a brand perspective.And just in case we refer to it so people know, and then there's Waymo Via, which is the trucking component. Why Via, by the way? What does that, what does that, what, is it just like a cool sounding name that just-
- BSBoris Sofman
Yeah.
- LFLex Fridman
... uh, like, is there a, is there an interesting story there or just, it is a pretty cool sounding name.
- BSBoris Sofman
It's a cool sounding name. I mean, when you think about it, it's just like, well, we're gonna transport it via this and that and that.
- LFLex Fridman
Oh, cool.
- BSBoris Sofman
So it's just kind of like an allusion to, um, the mechanics of transporting something.
- LFLex Fridman
Yes, cool.
- BSBoris Sofman
Um, and, uh, and it is a pretty good grouping, and the interesting thing is that even the groupings kind of blur where Waymo One is, like, human transportation, and, uh, there's a fully autonomous service in the Phoenix area that, like, every day is transporting people, and it's pretty incredible to, like, just see, you know, see that operate at reasonably large scale and just kind of happen. And then on the Via side, it doesn't even have to be, uh, like, long-haul trucking is a, like, a, a major focus of, uh, of ours. But down the road, you can stitch together the vehicle transportation as well for local delivery, um, also. And a lot of these requirements for local delivery overlap very heavily with consumer transportation, um, o- obviously, uh, you know, given that you're operating on a lot of the same roads, um, and, uh, uh, and s- navigating the same safety challenges. And so, um, yeah, and Waymo very much is a, uh, you know, multi-product company that, uh, has ambitions in both. They have different challenges, and both are tremendous opportunities. But the cool thing is, is that there's a huge amount of leverage, and this kind of core technology stack now gets pushed on by both sides.
- LFLex Fridman
Mm-hmm.
- BSBoris Sofman
Um, and that adds its own unique challenges. But the success case is that, um, the challenges that you push on, um, they get leveraged across all platforms and all domains.
- LFLex Fridman
So the, the, from an engineer perspective, the teams are integrated.
- BSBoris Sofman
It's a mix. So there's a huge amount of centralized kind of core teams that support all applications. And so you think of something like the hardware team that develops the lasers to compute, integrates into vehicle platforms. This is an experience that carries over across, um, you know, any application that we have, and they ebb and flow with both. Then there's, like, really unique, um, perception challenges, planning challenges, like, other, you know, types of challenges where there's a huge amount of leverage on a cortex stack, but then there's, like, dedicated teams that think of, how do you deal with a unique challenge? For example, um, an articulated trailer with varying loads that completely changes the physical dynamics of a vehicle. That doesn't exist on a car, but it becomes one of the most important, um, kind of unique new challenges on a, on a truck.
Episode duration: 3:14:13
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode U_AREIyd0Fc
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome