Lex Fridman PodcastDmitri Dolgov: Waymo and the Future of Self-Driving Cars | Lex Fridman Podcast #147
EVERY SPOKEN WORD
150 min read · 30,008 words- 0:00 – 2:16
Introduction
- LFLex Fridman
The following is a conversation with Dmitry Dolgov, the CTO of Waymo, which is an autonomous driving company that started as Google Self-Driving Car Project in 2009, and became Waymo in 2016. Dmitry was there all along. Waymo's currently leading in the fully autonomous vehicle space, in that they actually have an at-scale deployment of publicly accessible autonomous vehicles driving passengers around with no safety driver, with nobody in the driver's seat. This, to me, is an incredible accomplishment of engineering on one of the most difficult and exciting artificial intelligence challenges of the 21st century. Quick mention of a sponsor, followed by some thoughts related to the episode. Thank you to Trial Labs, a company that helps businesses apply machine learning to solve real-world problems. Blinkist, an app I use for reading through summaries of books. BetterHelp, online therapy with a licensed professional. And Cash App, the app I use to send money to friends. Please check out these sponsors in the description to get a discount and to support this podcast. As a side note, let me say that autonomous and semi-autonomous driving was the focus of my work at MIT, and is a problem space that I find fascinating and full of open questions from both a robotics and a human psychology perspective. There's quite a bit that I could say here about my experiences in academia on this topic that revealed to me, let's say, the less admirable sides of human beings. But I choose to focus on the positive, on solutions, on brilliant engineers like Dmitry and the team at Waymo, who work tirelessly to innovate and to build amazing technology that will define our future. Because of Dmitry and others like him, I'm excited for this future. And who knows? Perhaps I, too, will help contribute something of value to it. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter @LexFridman. And now, here's my conversation with Dmitry Dolgov.
- 2:16 – 7:23
Computer games
- LFLex Fridman
When did you first fall in love with robotics, or even computer science, more in general?
- DDDmitri Dolgov
Computer science first, at a fairly young age. Robotics happened much later. Um, I, uh, I think my first interesting introduction to computers was in the late '80s, uh, when we got our first computer. I think it was an, uh, an IBM, I think. IBM AT, I think. Remember those things that had, like, a turbo button in the front?
- LFLex Fridman
Turbo button? Yeah.
- DDDmitri Dolgov
Where, where you would press it and, you know, make, make the thing goes faster.
- LFLex Fridman
Did they already have floppy disks?
- DDDmitri Dolgov
Yeah. Yeah, yeah. Yeah, like the, the 5.4-inch ones.
- LFLex Fridman
I think there was a bigger inch. So good- went something, then five inches, then three inches.
- DDDmitri Dolgov
Yeah, I think that was the five. I don't, I, maybe that was before that was the, the giant plates, and I didn't get that. Uh, but it was definitely not the, not the three-inch ones. Uh, anyway, so that, that, you know, we got that, uh, computer. I spent the first, uh, few months just, you know, playing video games, uh, as you would expect. I, uh, got bored of that, uh, so I, uh, started messing around and, uh, trying to figure out how to, you know, make the thing do other stuff. Got into, uh, exploring, you know, programming, and a couple of years later, it got to a point where, um, I actually wrote a game, uh, a, a little game.
- LFLex Fridman
Nice.
- DDDmitri Dolgov
And a game developer, uh, a Japanese game developer actually offered to buy it from me for, you know, a few hundred bucks, but, you know, for, for a kid, uh, in Russia, uh-
- LFLex Fridman
It's a big deal.
- DDDmitri Dolgov
It's a big deal, yeah. (laughs) Uh, I did not take the deal.
- LFLex Fridman
Wow, integrity.
- DDDmitri Dolgov
Yeah. Uh, I, I instead, I-
- LFLex Fridman
Or stupidity.
- DDDmitri Dolgov
Yes, that was not the most acute financial move that I made in my life, you know, looking back at it now. Uh, I, I instead put it, well, you know, I had a reason. I, I put it online. Uh, it was, what d- what do you call it back in the days? It was a freeware thing, right? It was not open source, but you could upload the binaries, you would put the game online, and the idea was that, you know, people like it, and then they, you know, contribute and they send you little donations, right? So I did my quick math of, like, you know, my, of course, you know, thousands and millions of people are gonna play my game, send me a couple bucks apiece, you know, should definitely do that. As I said, not- (laughs)
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... not the best financial decision of my life.
- LFLex Fridman
You're already playing with business models at that young age.
- DDDmitri Dolgov
Yeah, yeah.
- LFLex Fridman
Remember what language it was? What programming lang- was it Basic?
- DDDmitri Dolgov
Oh, Pascal.
- LFLex Fridman
Was what?
- DDDmitri Dolgov
Pascal.
- LFLex Fridman
Pascal. And it had a graphical component?
- DDDmitri Dolgov
It did.
- LFLex Fridman
So it's not text-based?
- DDDmitri Dolgov
Yeah, yeah. It was, uh, like, uh, I think, you know, 300, 320 by 200, uh, whatever it was. I think the, kind of the earlier version.
- LFLex Fridman
That's the resolution?
- DDDmitri Dolgov
It was a VGA resolution, right? And I, I actually think the reason why this c- company wanted to buy it is not, like, the fancy graphics or-
- 7:23 – 9:55
Childhood
- LFLex Fridman
did you have any ridiculously ambitious dreams of where as a creator you might go, as an engineer? Did you... What, what did you think of yourself as, as an engineer, as a tinkerer, or did you want to be like an astronaut or something like that?
- DDDmitri Dolgov
(laughs) You know, I'm tempted to make something up about, you know, robots, uh, engineering-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... or, you know, mysteries of the universe, but (laughs) that's not the actual memory that pops into my mind, uh, when you, when you asked me about childhood dreams. So I'll actually share the, the, the real thing. Uh, when I was maybe four or five years old, I, you know, as we all do, I, you know, thought about, you know, what I wanted to do when I grow up, and I had this dream of being a traffic control cop.
- LFLex Fridman
(laughs)
- DDDmitri Dolgov
Uh, you know, they don't have those today, so I think... But, you know, back in the '80s in, you know, in Russia, uh, you, you probably are familiar with that, Lex. They had these, uh, you know, police officers that would stand in the middle of intersection all day, and they would have their like striped black, well, black and white batons that they would use to, you know, control the flow of traffic. And, you know, for whatever reason, I was strangely infatuated with this whole process, and like that, that was my dream. Uh, that's what I wanted to do (laughs) when I grew up. And, you know, my parents, uh, both physics profs by the way, I think were, you know, a little concerned, uh, with that level of ambition coming from their child-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... uh, at, you know, that age.
- LFLex Fridman
Well, that... It's an interesting... I don't know if you can relate, but I very much love that idea. I have a OCD nature that I think lends itself very close to the engineering mindset, which is you want to kind of optimize, you know, solve a problem by create- creating an automated solution, like a, like a s- a set of rules, the set of rules that you follow and then thereby make it ultra efficient. I don't know if that's... uh, it was i- of that nature. I, I certainly have the... There's like fact- like SimCity and factory building games, all those kinds of things kind of speak to that engineering mindset. Or did you just like the uniform?
- DDDmitri Dolgov
I think it was more of the latter. I think (laughs) it was the uniform-
- LFLex Fridman
(laughs)
- DDDmitri Dolgov
... and the, you know, the, the striped baton there that made cars go (laughs) in right directions that did drove me... But I guess, you know, I, it is... I did end up, uh, I guess, uh, you know, working in the transportation industry one way or another. (laughs)
- LFLex Fridman
No uniform though, but-
- DDDmitri Dolgov
Uh, that's right. That's right. So less-
- LFLex Fridman
... (laughs) batons.
- DDDmitri Dolgov
Maybe, may- maybe it was my, you know, deep inner infatuation with the, you know, traffic control batons that led to this, uh, career.
- 9:55 – 10:44
Robotics
- DDDmitri Dolgov
- LFLex Fridman
Okay. What, uh... When did you... When was the leap from programming to robotics?
- DDDmitri Dolgov
That happened later. That was after grad school, uh, after... And I actually, you know, was self-driving cars was, I think, my first real-
- LFLex Fridman
Real world.
- DDDmitri Dolgov
... hands-on introduction to robotics, yeah. But I, I never really had that much hands-on experience, you know, at school and training. I, you know, worked on applied math and physics. Then in, you know, college I did more kind of, uh, abstract, uh, computer science. Uh, and it was after grad school that I really got involved in robotics, which was actually self-driving cars, and, you know, that was a big, big flip. Uh-
- LFLex Fridman
What, uh, what grad school?
- DDDmitri Dolgov
So I went to grad school in Michigan, and then I did a post-doc at Stanford, uh, which is... That was the post-doc where I got to play with self-driving cars.
- LFLex Fridman
Yeah. So we'll return
- 10:44 – 12:56
Moscow Institute of Physics and Technology
- LFLex Fridman
there, but let's go back to, uh, to Moscow. So, I, uh... You know, for episode 100, I talked to my dad, and also I grew up with my dad, I guess. (laughs) Uh, so I, I had to put up with him for many years. And, uh, he, he went to the PHISTEH or MIPT. It's weird to say in English because I've heard all of this in Russian. Moscow Institute of Physics and Technology. And to me that was like I met some super interesting... As a child, I met some super interesting characters. It felt to me like the greatest university in the world, the most elite university in the world. And just the, the people that I met that came out of there were like not only brilliant but also special humans. It seems like that place really tested the soul, (laughs) uh, both like i- uh, in terms of technically and like spiritually. So that could be just the ro- romantization of that place, I'm not sure, but so maybe you can speak to it. But did... Is it correct to say that you spent some time at PHISTEH?
- DDDmitri Dolgov
Yeah, that's right, uh, six years. Uh, I got my bachelor's and master's in, uh, physics and math there. And it actually interesting because my, m- my dad, actually both my parents, uh, went there, and I think all the stories that I heard, uh, like, just like you, uh, Lex, uh, growing up about the place and, you know, how interesting and special and, you know, magical it was, I think that was a significant...... maybe the main reason, uh, I wanted to go there, uh, for college, uh, enough so that I actually went back to Russia from the US. I graduated high school in the US-
- LFLex Fridman
Mm-hmm.
- DDDmitri Dolgov
... um, and-
- LFLex Fridman
You went back there?
- DDDmitri Dolgov
I went back there, yeah. That-
- LFLex Fridman
Wow.
- DDDmitri Dolgov
... exactly the reaction most of my peers in college had, but, you know, perhaps a little bit stronger that like, they would, you know, point me out as this crazy kid who-
- LFLex Fridman
Were your parents supportive of that?
- DDDmitri Dolgov
Yeah. Yeah. Like, it was to your previous question, they, uh, they supported me and, you know, a lot of letting me kind of pursue my passions and the, you know, things that I was interested in.
- LFLex Fridman
That's a bold move. Wow. What was it like there?
- DDDmitri Dolgov
It was interesting, you know, definitely fairly hardcore on the fundamentals of, you know, math, uh, and physics and, uh, you know, lots of good memories, uh, from, you know, from those
- 12:56 – 23:16
DARPA Urban Challenge
- DDDmitri Dolgov
times.
- LFLex Fridman
So, okay. So Stanford, how'd you get into autonomous vehicles?
- DDDmitri Dolgov
I had the great fortune, uh, and great honor to join Stanford's DARPA Urban Challenge team in, uh, 2006. There, this was, uh, third in the sequence of the DARPA challenges. There were two grand challenges prior to that, and then in 2007, they held the DARPA Urban Challenge. So, you know, I was doing my postdoc I had, I joined the team and, uh, worked on motion planning, uh, for, you know, that, that competition.
- LFLex Fridman
So, okay. So for people who might not know, I know from, from a certain... (laughs) Autonomous vehicles is a funny world, i- in a certain circle of people, everybody knows everything. And then in a certain circle, uh, nobody knows anything, I think in terms of general public. So it's interesting. It's, it's a good question what to talk about, but I do think that the urban challenge is worth revisiting. It's a fun little challenge. One that inspi- first, like sparked so much, so many incredible minds to focus on one of the hardest problems of our time in artificial intelligence. So that's, that's a success from a perspective of a single little challenge. But can you talk about, like, what did the challenge involve? So were there pedestrians? Were there other cars? What was the goal? Uh, who was on the team? How long did it take? Any fun, fun sort of specs? (laughs)
- DDDmitri Dolgov
Sure, sure, sure. So the way the, the challenge was constructed and just a little bit of background, as I mentioned, this was the third, uh, competition in that series. The first two, though, were the Grand Chal- called the Grand Challenge. The goal there was to just drive in a completely static environment, you know, you had to drive in a desert.
- LFLex Fridman
(gasps) .
- DDDmitri Dolgov
Uh, s- that was very successful. So then DARPA followed with what they called the Urban Challenge, where the goal was to ha- you know, build vehicles that could operate in more dynamic environments and, you know, share them with other vehicles. There were no pedestrians, uh, there. But what DARPA did is they took over an abandoned Air Force base, uh, and it was kind of like a little fake city, uh, that they built out there, and they had a bunch of, uh, robots, uh, you know, cars that were autonomous, uh, in there all at the same time, uh, mixed in with other vehicles driven by professional, uh, drivers. And each car, uh, had a mission. And so there's a crude, uh, map that they received, uh, at the beginning, and they had a mission, you know, go, you know, here and then there and over here. Um, and they kind of all were sharing this environment at the same time. They had interact- to interact with each other. They had to interact with the human drivers. So it's this very first, very rudimentary, um, version of, uh, a self-driving car that, you know, could operate, uh, and on, uh, in a, in an environment, you know, shared with other dynamic actors that as you said, you know, really, you know, in many ways, you know, kickstarted this whole industry.
- LFLex Fridman
Okay. So who was on the team and how'd you do? I forget.
- DDDmitri Dolgov
(laughs) Uh, we came in second.
- LFLex Fridman
(laughs)
- DDDmitri Dolgov
Uh, perhaps that was my contribution to the team.
- LFLex Fridman
(laughs)
- DDDmitri Dolgov
I think the Stanford team came in first in the DARPA challenge, uh, but then I joined the team and, you know, we-
- LFLex Fridman
You were the one with the bug in the code.
- DDDmitri Dolgov
I, I-
- LFLex Fridman
I mean, do you have sort of memories of some particularly challenging things or, you know, one of the cool things it's not a ... you know, this isn't a product, this isn't a thing that, uh, you know ... it there's ... you have a little bit more freedom to experiment so you can take risks and there's, uh ... so you can make mistakes. Uh, so is there interesting mistakes, is there interesting challenges that stand out to you as something, like taught you, um, a, a good technical lesson or a good philosophical lesson from that time?
- DDDmitri Dolgov
Yeah. Uh, you know, definitely, definitely a very memorable time. Not really a challenge, but like, uh, one of the most vivid memories that I have from the time, and I think that was actually one of the days that, you know, really got me hooked, uh, on this whole field was, uh, the first time I got to run my software on the car, and, uh, I was working on a part of our planning algorithm, uh, that had to navigate in parking lots. So it was, you know, something that, you know, called free space motion planning. So the very first version of that, uh, wa- you know, we tried on the car, it was on Stanford's campus, uh, in the middle of the night, and you know, had this little, you know, course constructed with cones, uh, in the middle of a parking lot. So we're there at like 3:00 AM you know, by the time it got the code to, you know, uh, uh, you know, compile and turn over. Uh, and, you know, it drove. Like, it actually did something quite reasonable and, you know, it was of course very buggy at the time and had all kinds of problems, but it was pretty darn magical. I remember going back and, you know, you know, late at night and trying to fall asleep and just, you know, being unable to fall asleep for, you know, the rest of the night. Uh, just my mind was blown, just like ... and yeah, that, that, that's what I've been, you know, doing ever since for, you know, more than a decade. Uh, in terms of challenges and, uh, you know, it's interesting memories, like on the day of the competition, uh, it was, you know, pretty nerve-wracking. Uh, I remember, you know, standing there with Mike Montemerlo, who was, uh, the software lead and wrote most of the code. Like, I did one little part of the planner. Mike, you know, incredibly t-... did, you know, pretty much the rest of it, uh, with- with- with, you know, a bunch of other incredible people. But I remember standing on the day of the competition, uh, you know, watching the car, you know, with Mike, and, you know, cars are, uh, completely empty, right? They're all there lined up in the beginning of the race. And then, you know, DARPA sends them, you know, o- on their mission one by one, so then leave. And, like, you just, they had these sirens, wam, wam, wam. They all had their different siren- sirens, right? Each siren had its own personality, if you will. So, you know, off they go, and you don't see them, you just kind of... And then every once in a while, they, you know, come a little bit closer to where, uh, the audience is, and you can kinda hear, you know, the- the sound of your car, and, you know, it seems to be moving along so that, you know, it gives you hope. And then, you know, it goes away and you can't hear it for too long, you start getting anxious, right? So it's a little bit like, you know, sending your kids to college and like, you know, kind of you invested in them, you hope you- you- you- you- you- you build it properly, but, like, it's- it's still, uh, anxiety inducing. Uh, so that was, uh, an incredibly, uh, fun, uh, few days. In terms of, you know, bugs, as we mentioned, you know, one that- that was my bug that caused us the loss of the first place.
- LFLex Fridman
Mm-hmm.
- DDDmitri Dolgov
Uh, is still, uh, a debate that, you know, I occasionally have with people on the CMU team. CMU came first, I- I should mention, uh, that-
- LFLex Fridman
CMU, haven't heard of them, but yeah.
- DDDmitri Dolgov
No, it's some, you know-
- LFLex Fridman
It's a small school somewhere.
- DDDmitri Dolgov
... little, little school. It's- it's- it's, yeah, really a glitch that, you know, they happen to succeed at something robotics related.
- LFLex Fridman
Very scenic though, so, you know, most people go there for the scenery. Um, yeah.
- DDDmitri Dolgov
That's right.
- LFLex Fridman
It's a beautiful campus. (laughs) I apologize.
- DDDmitri Dolgov
Unlike- unlike Stanford.
- LFLex Fridman
So for people... Yeah, that's true, unlike Stanford. For people who don't know, CMU is one of the great robotics and sort of artificial intelligence universities in the world. CMU, Carnegie Mellon University. Okay, sorry, go ahead.
- DDDmitri Dolgov
Good- good PSA. So in the part that I contributed to, which was navigating parking lots, and the way, you know, that part of the mission worked is, uh, you, in a parking lot, you would get from DARPA an outline of the map. You basically get this, you know, giant polygon that defined the perimeter of the parking lot, uh, and there would be an entrance and, you know, so maybe, you know, multiple entrances or exits to it, and then you would get a goal, uh, within that open space, uh, X, Y, you know, heading, where the car had to park. It had no information about the opticals- obstacles that the car might encounter there, so it had to navigate a kind of completely free space, uh, from the entrance to the parking lot into that parking space, and then, uh, once, you know, parked there, it had to, uh, exit the parking lot, you know, while of course counting and reasoning about all the obstacles that it encounters in real time. So, uh, m- our interpretation, or at least my interpretation of the rules was that you had to reverse out of the parking spot, and that's what our cars did, even if there's no obstacle in front. That's not what CMU's car did, and it just kind of dr- drove right through. So there's still a debate and of course, you know, if you stop and then reverse out and go at the different way, that costs you some time, right? So there's still a debate whether, you know, it was my poor implementation that cost us extra time or whether it was, you know, CMU, uh, violating an important rule of the competition. And, you know, I have my own, uh, uh, opinion here. In terms of other bugs, and like, uh, I- I have to apologize to Mike Montemerola, uh, for sharing this on air.
- LFLex Fridman
(laughs)
- 23:16 – 38:58
Waymo origin story
- LFLex Fridman
So can we, uh, give an overview of how was Waymo born? How was the Google self-driving car project born? What is the- what is the mission? What is the hope? What is- it is the engineering kind of, uh, set of milestones that it seeks to accomplish? There's a lot of questions in there.
- DDDmitri Dolgov
Uh, yeah. (laughs)
- LFLex Fridman
I don't know.
- DDDmitri Dolgov
You're right, it- it- it kind of the DARPA Urban Challenge and the DARPA- previous DARPA Grand Challenges, uh, kind of led, I think, to a very large, you know, degree to that next step and then Larry and Sergey, um, Larry Page and Sergey Brin, Google founders, of course, saw that competition and, you know, believed in the technology. So, you know, the Google self-driving car project was born. You know, at that time, and we- we started in 2009, it was a pretty small group of us, about a dozen people who came together, uh, to- to work on- on this project at Google. At that time, we saw an, you know, that incredible early result in the DARPA Urban Challenge. I think we were all incredibly excited, uh, about-... w- where we got to, and we believed in the future of the technology, but we still had a very, uh, rudimentary understanding of the problem space. So, the first goal of this project in 2009 was to really better understand what we're up against. Uh, and, you know, with that goal in mind, when we started the project, we created a few milestones for ourselves, uh, that maximized learnings, if you will. The- the two milestones were, you know, uh, one was to drive 100,000 miles in autonomous mode, which was, at that time, you know, orders of magnitude that, uh, more than anybody has ever done. And the second milestone was to drive 10 routes. Uh, each one was 100 miles long, and they were specifically chosen to be kind of extra spicy, you know, extra complicated and s- sample the full complexity-
- LFLex Fridman
Right.
- DDDmitri Dolgov
... of the, that, that, uh, domain. Um, uh, and you had to drive each one from beginning to end, with no intervention, no human intervention. So you would get to the beginning of the course, uh, you would, you know, press the, the button that would engage in autonomy, and you had to, you know, go for 100 miles, you know, beginning to end, uh, with no interruptions. Um, and it sampled, again, the full complexity of driving conditions. Some, uh, were on freeways, we had one route that went all through all the freeways and all the bridges in the Bay Area. You know, we had, uh, some that went around Lake Tahoe and kind of mountainous, uh, roads. We had some that drove through dense urban, um, environments, like in downtown Palo Alto and through San Francisco. So, it was incredibly, uh, interesting, uh, to work on. And it, uh, it took us just under two years, uh, about a year and a half, little bit more, to finish both of these milestones. And in that process, uh, you know, A, it was an incredible amount of fun, probably the most fun I had in my professional, you know, career. And 'cause you're just learning so much, you are, you know, the goal here is to learn and prototype, you're not yet starting to build a production system.
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
Right? So you just, you were, you know, this is when you're kind of, you know, working 24/7 and, you know, hacking things together.
- LFLex Fridman
And you also don't know how hard this is. I mean, that's the point. Like, so, I mean, that's an ambitious, if I put myself in that mindset, even still, that's a really ambitious set of goals. Like, just those two, just picking, uh, s- picking 10 different difficult, spicy challenges, and then having zero interventions. So like, not saying, "Gradually, we're going to, like, you know, over a period of 10 years, we're going to have a bunch of routes and gradually reduce the number of interventions." You know, that literally says, like, "By as soon as possible, we wanna have zero, and on hard roads." So, so like, to me, if I was facing that, it's unclear that whether that takes two years or whether that takes 20 years. I mean, it may be-
- DDDmitri Dolgov
It took us under two. And I guess that, that speaks to the, a really big difference between doing something once and having a prototype-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... uh, where you are going after, you know, learning about the problem, versus how you go about engineering a product that, you know, where you, uh, look at, uh, you know, you properly do evaluation, you look at metrics, you, you know, drive down, and you're confident that you can do that 100. And I guess that's the, why it took, uh, a dozen people, uh, you know, 16 months or a little bit more than that, uh, back in 2009 and 2010, with the technology of, you know, the, more than a decade ago, uh, that amount of time, to achieve that milestone of, you know, 10 routes, uh, 100 miles each and no interventions. Uh, and, you know, it took us a little bit longer to get to, you know, a full driverless product-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... uh, that customers use.
- LFLex Fridman
That's another really important moment. Is there some memories of, uh, technical lessons or just, one, like, what did you learn about the problem of driving from that experience? I mean, we can, we can now talk about like what you learned from modern day Waymo, but I feel like you may have learned some profound things in those early days even more so, because it feels like what Waymo is now is they're trying to, you know, how to do scale, how to make sure you create a product, how to make sure it's like safety and all those things, which is all fascinating challenges. But like, you were facing the more fundamental philosophical problem of driving in those early days, like, what the hell is driving, uh, as an autonomous v- or maybe I'm, again, romanticizing it, but is (laughs) , is there, uh, is there some valuable lessons you picked up over there at th- those two years?
- DDDmitri Dolgov
Uh, a ton. The most important one is probably that we believe that it's doable.
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
And we- we've gotten, uh, far enough into the problem that, uh, you know, we had a, I think only a glimpse of the true complexity, uh, of the- the- the- the domain. You know, it's a little bit like, you know, climbing a mountain where you kind of, you know, see the next peak and you think that's kind of the summit, but then you get to that and you kind of see that- that this is just the- the start of the journey. Uh, but we've tried, we've sampled enough of the problem space and we've made enough rapid, uh, success even, you know, with technology of 2009, 2010, that, uh, it gave us confidence to then, you know, pursue this as a real product. So...
- LFLex Fridman
Okay, so the next step. You mentioned the- the milestones that you had in the, in the, in those two years. What are the next milestones that then led to the creation of Waymo and beyond?
- DDDmitri Dolgov
Yeah, we had a- a- it was a really interesting journey and, you know, Waymo came a little bit later, uh, than, you know, the... We completed those milestones in 2010. Uh, that was the pivot when we decided to focus on actually building a product, you know, using this technology. Uh, the initial couple of years after that, we were focused on a freeway, you know, what you would call a driver assist, uh, maybe on, you know, on all three, uh, driver assist, uh, program. Then around 2013, we've learned enough, uh, about the space and had thought, you know, more deeply about, you know, the product that we wanted to build that we pivoted it. Uh, we pivoted towards, uh, this vision of, you know, building a driver, uh, and deploying it fully driverless vehicles without a person, and that- that's the path that we've been on since then, and, uh, very... it was exactly the right decision for us.
- LFLex Fridman
So there was a moment where you also considered, like, what is the right trajectory here? What is the right role of automation in the- in the task of driving? There's still, uh, it- it wasn't from the early days, obviously, you wanna go fully autonomous.
- DDDmitri Dolgov
From the early days, it was not. I think it was in 20- around 2013 maybe, uh, that we've... that became very clear and we made a pivot, and it also became very clear, uh, and that it's... the way you go building a driver assist system is, you know, fundamentally different from how you go building a fully driverless vehicle. So, you know, we've, uh, pivoted, uh, towards the latter and that's what buil- we've been working on ever since. And so that was around 2013. Then, uh, there's a sequence of really, um, meaningful for us, really important, uh, defining milestones since then and, uh, 2015, we, uh, had our first, actually the world's first fully driverless, uh, ride on, uh, public roads. It was in a custom-built vehicle that we had.
- LFLex Fridman
Mm-hmm.
- DDDmitri Dolgov
You must have seen those. We called them the Firefly that, you know, funny looking marshmallow-looking thing. Um, and we, uh, put, uh, a passenger, uh, his name was Steve Mann, you know, great, uh, friend of our project from the early days, uh, the- the man happens to be, uh, blind. So we put him in that vehicle. Uh, the car had no steering wheel, no pedals. It was an uncontrolled environment. Um, you know, no, you know, lead or chase cars, no police escorts. Um, and, uh, you know, we did that trip a few times in Austin, Texas. So that was a- a really big milestone.
- LFLex Fridman
Oh, that was in Austin?
- DDDmitri Dolgov
Yeah.
- LFLex Fridman
Cool. Okay.
- DDDmitri Dolgov
Yeah. Um, and, you know, we only... but at that time, we're only... it- it took a tremendous amount of engineering. It took a tremendous amount of validation, uh, to get to that point. Um, but, you know, we only did it a few times, right? We only did that... It was a fixed route. It was not kind of a controlled environment, but it was a- a fixed route and we only did it a few times. Uh, then, uh, in, uh, 2016, uh, end of 2016, beginning of 2017 is, uh, when we founded Waymo, uh, the company. That's when, you know, we, uh, kind of that was the next phase of the project where I wanted... uh, we believed in kind of the commercial, uh, vision of this technology. And it made sense to create an independent entity, you know, within that Alphabet, uh, umbrella, to pursue, uh, this product, um, at scale. Uh, beyond that in 2017, later in 2017, uh, was another really, uh, huge step for us, really big milestone where we started, I think it was October of 2017, where- when we, uh, started regular driverless operations on public roads. Uh, that first day of operations, we drove, uh, in one day, in that first day, 100 miles in, you know, driverless fashion, and then we f- uh... The most- the most important thing about that milestone was not that, you know, 100 miles in one day, but that it was the start of kind of regular ongoing driverless operations.
- LFLex Fridman
And when you say driverless, it means no driver?
- DDDmitri Dolgov
That's exactly right. So on that first day, we actually had a mix and up... uh, in some, uh... we didn't want to like, you know, be on YouTube and Twitter that same day so in, uh, in s- in many of the rides, we had somebody in the driver's seat-
- 38:58 – 47:31
Waymo self-driving hardware
- DDDmitri Dolgov
Uh, we've also, you know, uh, started putting our fifth generation of our driver, our hardware, uh, uh, that is on the new vehicle, but it's also a qualitatively different set of, uh, self-driving hardware, uh, that's all dr- uh, that, uh, is now on the JLR pace, so that was a very important step for us. The-
- LFLex Fridman
Hardware specs, fifth generation, I think it'd be fun to maybe, uh, I apologize if I'm interrupting, but, uh, maybe talk about maybe the generations with a focus on what w- we're talking about in the fifth generation in terms of hardware specs. Like, what's on this car?
- DDDmitri Dolgov
Sure. So we separated out, you know, the actual car that we are driving from the self-driving hardware we put on it.
- LFLex Fridman
(laughs)
- DDDmitri Dolgov
Um, right now we have, so this is, as I mentioned, the fifth generation. You know, we've gotten, like, uh, through ... We, we started, you know, building our own hardware, you know, many, many years ago, and, uh, that, you know, Firefly vehicle, uh, also had the hardware suite that was mostly, you know, designed and engineered and built in-house. Uh, LIDARS are of, one of the more important, uh, components that we design and build from the ground up. Uh, so on the fifth generation, uh, of their, uh, drivers, uh, of our self-driving hardware that we're, uh, switching to right now, uh, we have, uh, as with previous generations, in terms of sensing, we have LIDARS, cameras and radars. And we have a pretty beefy computer that processes all that information and makes, you know, decisions in real time on, onboard the car. Uh, so in all of the ... And it, it's really a qualitative, uh, jump forward in terms of the capabilities and, uh, various parameters and specs of the hardware compared to what we had before, and, you know, compared to what you can kind of get off the, off the shelf in the market today. Uh-
- LFLex Fridman
Meaning from fifth to fourth or from fifth to first?
- DDDmitri Dolgov
Definitely (laughs) from, uh, first to fifth, but also from the fourth.
- LFLex Fridman
That was the world's dumbest question. Okay. (laughs)
- DDDmitri Dolgov
D- definitely, no, uh, definitely from fourth to fifth.
- LFLex Fridman
Okay.
- DDDmitri Dolgov
Uh, as well as, uh, uh, just the, the, that last step is a, is a big step forward.
- LFLex Fridman
So everything's in-house. So the, like, LIDAR is built in-house?
- DDDmitri Dolgov
Mm-hmm.
- LFLex Fridman
And, and cameras are built in-house?
- DDDmitri Dolgov
Uh, you know, it's different-
- LFLex Fridman
Or is that s-
- DDDmitri Dolgov
You know, we work with partners.
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
There's some components, uh, that, you know, we, you know, get from our manufacturing and, you know, supply chain partners. Uh, what exactly is in-house is a bit different. If you ... Like, we, we do a lot of, you know, custom, uh, design on all, uh, of our sensing
- NANarrator
Okay.
- DDDmitri Dolgov
... modularies?, LIDARS, radars, cameras, you know, exactly there's, uh, LIDARS are, y- almost exclusively, uh, in-house. And some of the technologies that we have, some of the fundamental technologies there are completely unique, uh, to Waymo. Uh, that is also largely true about radars, and cameras it's a little bit more of a, a mix w- in terms of what we do ourselves versus what we get from, uh, partners.
- LFLex Fridman
Is there something, uh, super sexy about the computer that you can mention that's not top secret? Like, uh, (laughs) for, for people who enjoy computers for ... I mean, uh, so y- there's, there's a lot of, uh, machine learning involved, but there's a lot of just basic computers. You have to, uh, probably do a lot of signal processing on all the different sensors. You have to integrate, everything has to be in real time. There's probably some kind of redundancy type of situation.Is there something interesting you could say about the computer for the people who love hardware?
- DDDmitri Dolgov
It does have all of the characteristics, all the properties that you just mentioned. Uh, redundancy, uh, very beefy compute, uh, for just general processing as well as, you know, inference and ML models. It is some of the more sensitive stuff that, you know, I don't wanna get into for IP reasons. But, uh, it can... we sh- we've shared a little bit, uh, in terms of the specs of the sensors, uh, that we have on the car. You know, we actually shared, you know, some videos of, uh, what our lidar sees, uh, lidars see in the world.
- LFLex Fridman
Mm-hmm.
- DDDmitri Dolgov
Uh, we have 29 cameras. We have, you know, five lidars. We have six radars, uh, on these vehicles. And you can kind of get a feel for the amount of data that they're producing that all has to be processed in real time, uh, to, you know, do perception, to do complex reasoning. So it kind of gives you some idea of how beefy those computers are. But, you know, I don't wanna get into specifics of exactly how we build them.
- LFLex Fridman
Okay. Well, let me try some more questions that you can get into the specifics of. Like, GPU-wise, is that something you can get into? You know, I know that Google works th- with GPUs and so on. I mean, for machine learning folks, it's kind of interesting or is there no... How do I ask it? Uh, I've been talking to people in the government about UFOs, and they don't want to answer any questions.
- DDDmitri Dolgov
(laughs)
- LFLex Fridman
So this is, this is how I feel right now asking about GPUs. (laughs)
- DDDmitri Dolgov
(laughs)
- LFLex Fridman
But is there something interesting that you could r- reveal? Or is it just, you know, um, uh, or would leave it up to our imagination some of, some of the compute? Is there any, I guess, is there any fun trickery? Like I, I talked to, uh, Chris Lattner for a second time, and he was a key person about TPUs and there's a lot of fun stuff going on in, in Google in terms of, uh, hardware that, uh, that optimizes for machine learning. Is there something you can reveal in terms of how much... you mentioned customization, how much customization there is for hardware for machine learning purposes?
- 47:31 – 53:23
Connected cars
- DDDmitri Dolgov
- LFLex Fridman
Okay. That first piece that you mentioned-
- DDDmitri Dolgov
Mm-hmm.
- LFLex Fridman
... that c- cars communicating to each other essentially, I mean, through perhaps through a centralized point. But what, uh... that's fascinating too. How much does that help you? Like, if you imagine, like, you know, right now the number of Waymo vehicles is whatever, X. I don't know if you can talk to what that number is, but it's- it's not in the hundreds of millions yet. And (laughs) imagine if the whole world is Waymo vehicles. Uh, like, that changes potentially the power of connectivity. Like, the more cars you have... I guess actually if you look at Phoenix 'cause there's enough vehicles, uh, there's enough... wh- when there's a s- like some level of density, you can start to probably do some really interesting stuff with the fact that cars can negotiate, can be, uh, can communicate with each other and thereby make decisions. Is there something interesting there that- that you can talk to about? Like how does that help with the driving problem from... as compared to just a single car solving the driving problem by itself?
- DDDmitri Dolgov
Uh, yeah, it's a, it's a spectrum. Uh, yeah, first I say that, you know, it's ... It, it helps, uh, and it helps in various ways but it's not required. Uh, right now the, the way we build our system, like each cars can operate independently. They can operate with no connectivity. Uh, so it, I think it is important that, you know, you have a fully, uh, autonomous, you know, fully capable, uh, driver, uh, that, you know, computerized driver, that each car has. Uh, then, you know, they do share information, uh, and they share information in real time and it really, really helps. All right, so the way we, uh, do this today is, uh, you know, whenever one car, uh, encounters something interesting in the world, whether, you know, it might be an accident or a new construction zone, that information immediately gets, uh, you know, u- uploaded over the air and is propagated to the rest of the fleet. So, and th- that's kind of how we think about maps as, uh, priors in terms of the knowledge of our, uh, drivers, uh, of our fleet of drivers, um, that is, you know, distributed across the fleet and, and it's updated, uh, in real time. So that, that's one use, uh, case. Uh, you know, you can imagine as the, you know, the, the density of these vehicles go up that they can exchange more information in terms of what they're planning to do, uh, and, uh, start, uh, influencing how they interact with each other, uh, as well as, you know, potentially sharing some observations, right, to help with, you know, if you have enough density of these vehicles where, you know, one car might be seeing something that another ... is relevant to another car, uh, that is very dynamic. You know, it's not part of kind of your updating your static prior, uh, of the map of the world, but it's more of a dynamic information that could be relevant to the decisions that another car is making in real time so you can s- see them exchanging that information and you can build on that. But again, uh, I, I see that as, uh, an advantage, but it's, you know, not a requirement.
- LFLex Fridman
So what about the human in the loop? So, uh, when I got a chance to drive with, uh, a ride in the, in the Waymo, uh, you know, there's customer service. (laughs) So, like, there is somebody that's able to, uh, dynamically, like, tune in and, uh, help you out. What, uh, what role does the human play in that picture? That's a fascinating, like, you know, the idea of teleoperation, being able to remotely control a vehicle. So here what we're talking about is like, (sighs) like frictionless, uh, like a human being able to in a, in, in a frictionless way sort of help you out. I don't know if they're able to actually control the vehicle. Is that something you could talk to?
- DDDmitri Dolgov
Uh, yes.
- LFLex Fridman
Okay.
- DDDmitri Dolgov
Uh, to be clear, we don't do teleoperation. I'm gonna believe in teleoperation for various reasons. That's not what we have in our cars. Uh, we do as I mentioned have, you know, a version of, you know, customer support, uh, you know, we call it Live Help. In fact, we find it, that it's very, uh, important for our rider experience, especially if it's your first trip, you've never been in a, you know, fully driverless, rider-only Waymo vehicle. You get in, there's nobody there, right? So you can imagine having all kinds of, you know, questions in your head like, "How this thing works?" Uh, so we've put a lot of thought into-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... kind of guiding our, uh, uh, our riders, our customers through that experience, especially for the first time. They get some information on the phone, uh, uh, if the fully driverless vehicle is used to service their trip. Uh, when you get into the car, we have an in-car, you know, screen and audio that kind of guides them and explains, uh, what to expect. Uh, they also have a button that they can push that will conne- connect them to, you know, a real live human being that they can talk to, all right, about this whole process. So that's one aspect of it. Uh, there is, uh, you know, I should mention that there is, uh, uh, another function that, uh, humans provide, uh, to our cars, but it's not teleoperation. You can think of it a little bit more like, you know, fleet assistance, kind of like, you know, traffic control, uh, that, that, that you have where our cars, again, they're responsible on their own for making all of the decisions, all of the driving decisions that don't require connectivity. They ... You know, anything that is safety or latency critical, uh, is done, you know, purely autonomously by onboard, uh, uh, our own onboard system. Uh, but there are situations where, you know, if connectivity is available, uh, and a car encounters a particularly challenging situation, you can imagine like a super hairy, uh, scene of an accident, uh, the cars will do their best. They will recognize that it's an off nominal situation. They will, you know, do their best to come up, you know, with the right interpretation and the best course of action in that scenario. But if a connectivity is available, they can ask, uh, for confirmation from, you know, a human mode, human, um, uh, uh, assistant there to kind of confirm those actions and, you know, perhaps, uh, provide a little bit of kind of contextual information and guidance.
- 53:23 – 57:45
Waymo fully driverless service in Phoenix
- DDDmitri Dolgov
- LFLex Fridman
So October 8th was when you're talking about the ... was Waymo launched the, the, the fully self, uh, the public version of its fully driverless – that's the right term I think – service in Phoenix. Is that October 8th?
- DDDmitri Dolgov
That's right. No, it was the-
- LFLex Fridman
Okay.
- DDDmitri Dolgov
... introduction of fully driverless rider-only vehicles into our, you know, public Waymo One service.
- LFLex Fridman
Okay, so that's, that's amazing. So it's like anybody can get into a Waymo in Phoenix?
- DDDmitri Dolgov
Uh, that's right. Uh, so we, uh, previously had, uh, early, uh, people in our early rider program, uh, taking fully driverless rides in Phoenix, and, uh, just, uh, this, uh, uh, a little while ago, we opened on October 8th, we opened that mode of operation to the public. So yeah, you can, you know, download the app and, you know, go on a ride. And there is, uh, a lot more demand right now, uh, for that service (laughs) than we have capacity, uh, so we're kind of, uh, managing that, but that's exactly the way you described it.
- LFLex Fridman
Yeah. Well, this is interesting. So there's more demand than you can, you can handle? Like what, um, what has been the reception so far? Like what ... I mean, okay, so, you know, that's, uh, this is a, a product, right?... that's a whole nother discussion of, like, how compelling of a product it is. Great. But it's also, like, one of the most kinda transformational technologies of the 21st century. So there, uh, i- it's also like a tourist attraction (laughs) . Like, it's fun to, you know, to, to be a part of it. So it'd be interesting to see, like, what do, what do people say? What do people f- uh, what, what have been the feedback so far?
- DDDmitri Dolgov
You know, still early days, but so far, the feedback has been, uh, in- incredible, uh, incredibly positive. Uh, they, um, we ask them for feedback during the ride, we ask them for feedback, uh, after the ride, uh, as part of their trip. You know, we ask them some questions and we ask them to, you know, rate the performance of our driver. Um, most, by far, you know, most of our drivers give us plus five stars (laughs) in our app, uh, which is, uh, absolutely great to see. And, you know, that's ... And we're, they're also giving us feedback on, you know, things we can improve. Uh, and, you know, that's, that's one of the main reasons we're doing this as Phoenix. And, you know, over the last couple of years and every day today, uh, we are just learning a tremendous amount of new stuff from our users. There's, there's no substitute for, you know, actually doing the real thing, actually having a fully driverless product, uh, out there in the field with, you know, users, uh, that are actually, you know, paying us money to get from point A to point B.
- LFLex Fridman
So this is of, uh, legitimate ... Like, th- this is a paid service?
- DDDmitri Dolgov
That's right.
- LFLex Fridman
(inhales) And the idea is you use the app to go from point A to point B, and then what, what are the As? What are the f- what's the freedom of the, of the starting and ending places?
- DDDmitri Dolgov
It's an area of geography where that service is enabled. It's a, you know, decent size of geography of territory. It's actually larger than, kind of the size of San Francisco. Uh, and, you know, within that you have, you know, full freedom of, you know, selecting where you wanna go. You know, of course there are some ... And you, you on your app, uh, you get a map and you tell the car where you wanna be picked up, you know, and where you want, you know, you know, the car to pull over and pick you up, and then you tell it where you wanna be dropped off, right? And of course, there are some exclusions, right? Do you wanna be, you know, you, uh, where in terms of where the car is allowed to pull over, right? So, you know, that you can't do, but, you know, besides that, uh, it's-
- LFLex Fridman
Amazing.
- DDDmitri Dolgov
... it's not like a fixed ... Just would be very, I guess- (laughs) I don't know, maybe that was the question behind your question, (laughs) but it's not a, you know, preset set of, uh, you know, destinations.
- LFLex Fridman
Yeah, so I guess it's ... So within the geographic constraints, with that- within that area or anywhere el- i- it can be ... You can be picked up and dropped off anywhere?
- DDDmitri Dolgov
That's right. And, you know, people use them on like all kinds of trips. They ... We have ... And we have an incredible spectrum of riders. I mean, uh, I think the youngest actually have car seats in them and we have, you know, people taking their kids on rides. I think the youngest, uh, riders we had on our cars are, you know, one or two years old, you know, and the full spectrum of use cases. People can take them to, you know, schools, uh, to, you know, go grocery sto- shopping, to restaurants, to bars, you know, run errands, you know, go shopping, et cetera, et cetera. You can go to your office, right? Uh, like the full spectrum of use cases. And, uh, people, you know, use them in their daily lives to get around, uh, and we see all kinds of, you know, really in- uh, interesting, uh, use cases and that, that, that's providing us incredibly valuable experience, uh, that we would then, you know, use to improve our product.
- LFLex Fridman
So
- 57:45 – 1:05:58
Getting feedback from riders
- LFLex Fridman
as somebody who's been on ... Done a few long rants with Joe Rogan and others about the toxicity of the internet and the comments and the negativity in the comments, I'm fascinated by feedback. I, I believe that most people are good and kind and intelligent and can provide like, uh, even in disagreement, really fascinating ideas. So a- on a product side, it's fascinating to me, like, how do you get the richest possible user feedback, like to improve? What's, what are the channels that you use to measure? 'Cause like you're, you're no longer ... (sighs) That's one of the magical things about autonomous vehicles is it's not ... It d- like, it's frictionless interaction with the human. So like you don't get to ... You know, it's just giving a ride. So like how do you get feedback from people to, in order to improve?
- DDDmitri Dolgov
Uh, yeah. Uh, great question. V- various mechanisms. Uh, so as part of the normal flow, we ask people for feedback. They ... As the car is driving around, you know, we have on the phone and in the car – uh, we have a touch screen – uh, in the car, uh, you can actually click some buttons and provide, uh, real time feedback on how the car is doing, uh, and how the car is handling a particular situation, you know, both positive and negative. Uh, so that's one channel. Uh, we have, as we discussed, customer support or live help where, you know, if a customer wants to ... Has a question, uh, uh, or he has some sort of a concern, they can talk to a person, uh, in real time. So that, that is another mechanism that gives us, uh, feedback. Uh, at the end of a trip, you know, we also ask them how things went. They give us, uh, comments and, you know, a star rating.
- LFLex Fridman
Mm-hmm.
- DDDmitri Dolgov
And, you know, if it's ... Uh, we also, you know, ask them, uh, to, uh, uh, explain what, you know, went, w- went well and, you know, what could be improved. And, uh, we, we have, uh, our, our riders are providing very rich, uh, feedback there. Uh, wa- a large fraction is, uh, very passionate and very excited about this technology. So we get really good feedback. Uh, we also run, uh, UXR studies, right? You know, specific and that are kind of more, you know, go more in depth and we all run both kind of, uh, lateral and longitudinal studies, um, where we have, you know, deeper engagement, uh, with our customers. You know, we have our user experience research team.
- LFLex Fridman
Tracking over time, that's when you say about longitude. No, that's cool.
- DDDmitri Dolgov
That's, that's exactly right. And, you know, that's another really valuable, uh, feedback, uh, source of feedback. And, you ... Uh, we're just co- covering a tremendous amount, right? Uh, uh, people go grocery shopping and they like wanna load, you know, 20 bags of groceries in our cars and like that, that's one workflow that you maybe don't, you know, think about, uh, kind of, you know, getting just right when you're building the driverless product. Uh, I have people like, you know, who, uh, bike as part of their trip. So they, you know, bike somewhere, then they get in our cars, they take a pa- part of their bike, they load into our vehicle and then go ... And that, that's, you know, how they, you know, where we want to pull over and how that, you know, uh, get in and get out.Um, uh, process works, uh, provides us some very useful feedback. In terms of, you know, what makes a good, uh, pick-up and drop-off location, uh, we get really valuable feedback. Uh, and in fact, we had to, um, uh, do some really interesting work with, uh, high definition maps and, uh, thinking about walking directions. And if you imagine, you're in a store, right? In some giant space and then, you know, you wanna be picked up somewhere. Like, if you just drop a pin at your current location, which is maybe in the middle of a shopping mall, like, what's the best location for the car to come pick you up? And you can, you know, have simple heuristics where you just kind of take your, you know, Euclidean distance, uh, and find the nearest, uh, spot where the car can pull over that's closest to you. But oftentimes, that's not the most convenient one, you know, I have many anecdotes where that heuristic (laughs) breaks in horrible ways. Uh, one example, uh, that, you know, I often mention is somebody wanted to be, you know, uh, uh, dropped off, uh, in Phoenix, uh, and, you know, we... The car picked a location, uh, that was close, the closest to there, you know, where the pin was dropped, uh, on the map in terms of, you know, latitude and longitude. Uh, but it happened to be on the other side of a parking lot that had this row of cacti, and the poor person had to, like, walk all around the parking lot to get to where they wanted to be in 110 degree heat. So that, you know, that wasn't optimal. So then, you know, we took all, take all of these, um, all of that feedback from our users and, uh, incorporate it into our system and, you know, improve it.
- LFLex Fridman
Yeah, I feel like that, like, requires AGI to solve the problem of, like, when you're... which is a very common case. When you're in a big space of some kind, like apartment building, it doesn't matter, some- some large space, and then you call the, like, a Waymo from there, right? Like, any, whatever, doesn't matter, rideshare vehicle, and like, uh, where's the pin supposed to drop? I feel like that's... I- you don't think... I think that requires AGI. I'm gonna... you know, in order to solve... Okay, the alternative in- which I think the Google search engine is taught, is like there's something really valuable about the perhaps slightly dumb answer, but a really powerful one, which is like what was done in the past by others. Like, what was the choice made by others? That seems to be... Uh, like in terms of Google Search, when you have like billions of searches that you could- you could see which... Like, when they recommend what you might possibly mean, they suggest based on not some machine learning thing, which they also do, but like on what was successful for others in the past in finding a thing that they were happy with. Is that integrated at all with Waymo? Like, what- what pickups worked for others?
- DDDmitri Dolgov
It is. I- I think you're exactly right. So there's... Uh, real... It's an interesting problem. Uh, naive solutions, uh, have, uh, interesting failure modes. Uh, so there's definitely lots of things that, uh, can be done to improve, uh, and both learning from, you know, what works, what doesn't work and actually it'll heal from, you know, getting richer data and getting more information about the environment and, you know, uh, uh, richer maps. Uh, but you're absolutely right that there's something... Like, there's some properties of solutions that, uh, in terms of the effect that they have on users, some are much, much, much better than others, right?
- LFLex Fridman
Right.
- DDDmitri Dolgov
And predictability and understandability is important. So you can have maybe something that is not quite as optimal, but is very natural and predictable, uh, to the user and kind of works the same way, uh, uh, all the time. And that matters, that matters, uh, a lot for the user experience. And- and... But, you know, to get to the basics, the pretty fundamental property is that the car actually arrives where you told it to arrive. Like, you can always, you know, change it, see it on the map and you can move it around if you don't like it, and... But, like, that property that the car actually shows up-
- LFLex Fridman
Reliably.
- DDDmitri Dolgov
... on pin-
- LFLex Fridman
Yeah.
- DDDmitri Dolgov
... is critical, which, you know, where, uh, compared to some of the human, uh, driven-
- LFLex Fridman
Yes (laughs) .
- DDDmitri Dolgov
... analogs, I think, you know, you- you can have more predictability. It's actually... Uh, the fact, uh, if- if I have a, uh, my little- my little bit of a detour here, uh, I think the fact that it's, you know, your phone and the car is two computers talking to each other, uh, can lead to some really interesting things we can do in terms of the user interfaces. You know, both in terms of function, uh, like the car actually shows up exactly where you told it, uh, you want it to be, but also some, you know, really interesting things in the user interface. Like, as the car is driving, as you, you know, call it and it's on the way to come and pick you up, and of course you get the position of the car and the route on the map. Uh, but... And they actually follow that route, of course. Uh, but it can also share some really interesting information about what it's doing. So uh, uh, as you know, our cars, uh, as they are coming to pick you up, if it's come- if a car is coming up to a stop sign, it will actually show you that like it's there sitting because it's at a stop sign. Or a traffic light, it will show you that it's go- you know, sitting at a red light. So you know, the like little things, uh, right? Uh, but it- I find those little touch, uh, touches uh, really interesting, really magical and it's just, you know, little things like that that you can do to kind of delight your users.
- 1:05:58 – 1:11:49
Creating a product that people love
- LFLex Fridman
You know, this makes me think of, um... there's some products that I just love. Like, there's a... there's a company called Rev, uh, rev.com, where I like... For this podcast, for example, I can just drag and drop a video and then they do all the captioning. Uh, it's humans doing the captioning, but they connect you... They- they automa- automate everything of connecting you to the humans, and they do the captioning and transcription, and it's all effortless. And it like... I remember when I first started using them, it was like, "Life is good." Like, 'cause it was so painful to- to figure that out earlier. Uh, the same thing with something called Izotope RX, this company I use for cleaning up audio, like the sound cleanup they do. It's like drag and drop and it just cleans everything up very nicely. Uh, another experience like that I had with Amazon One Click purchase-... first time, I mean, o- other places do that now, but just the effortlessness of purchasing, making it frictionless. It kind of communicates to me, like, I'm a fan of design, I'm a fan of products, that you can just create a really pleasant experience, that the simplicity of it, the elegance, just makes you fall in love with it. So on the, do you think about this kind of stuff? I mean, we've been, the, it's exactly what we've been talking about, it's like the little details that somehow make you fall in love with a product. Is that ... W- w- we went from like urban challenge days, (laughs) where, where love was not part of the conversation probably, and to, to this point where there's a, where there's human beings and you want them to fall in love with the experience. Um, is that something you're trying to optimize for, try to think about? Like how do you, how do you create an experience that people love?
- DDDmitri Dolgov
Oh, absolutely. I think that's ... The vision is removing any friction or complexity from getting our users, our riders, to where they want to go, right? So making that as simple as possible, and then, you know, beyond that, and, uh, just transportation, making, you know, things and, you know, goods get to their destination as seamlessly as possible. I mean, talked about, you know, a drag-and-drop experience where you kind of express your intent and then, poof, you know, it just magically happens. And for our riders, that's what we're trying to get to is you download an app and you, you know, click and car shows up. It's the same car. It's very predictable. It's, you know, a s- a safe and high-quality experience, and then it gets you in a very reliable, very convenient, uh, frictionless way to where you want to be. And along the journey, I think we also want to like do little things to delight our, our users.
- LFLex Fridman
Like the ride-sharing companies, because they don't control the experience, I think, they can't make people fall in love necessarily with the experience, or maybe they, they haven't put in the effort, but I, I think it, if I were to speak to the ride-sharing experience I currently have, it's just very, it's just very convenient. But there's a lot of room for like falling in love with it. Like we- we can speak to sort of car companies. Car companies do this well. You can fall in love with a car, right? And be like a loyal car pers- like whatever. Like I like badass hot rods, like a '69 Corvette. And, and at this point, you know, you can't really ... cars are so ... owning a car is so 20th century man. But is there something about the Waymo experience where you hope that people will fall in love with it? Because that ... is that part of it or is it part of ... is it just about making a convenient ride ... not ride-sharing, I don't know what the right term is, but just a convenient A to B autonomous tra- um, transport? Or like do you want them to fall in love with Waymo? So maybe elaborate a little bit. I mean, almost like from a business perspective, I'm curious, like w- how ... do you want to be in the background invisible or do you want to be, uh, like a source of joy that's in the- very much in the foreground?
- DDDmitri Dolgov
I want to provide the best, most enjoyable transportation solution, and, uh, and that means building it, building our product and building our service in a way that people do, uh, kind of use in a very, your ... seamless, frictionless way in their, in their day-to-day lives. And I think that does mean, uh, you know, in some way falling in love, uh, in that product, right? It just kind of becomes part of your routine. I, uh, it, it comes down in my mind to, uh, safety, predictability of the experience, and, um, privacy, I think, uh, aspects of it, right? Uh, uh, our cars, uh, you get the same car. You get very predictable behavior, uh, and that, that is important, I think if you're gonna use it in your daily life. Uh, uh, privacy, I mean, when you're in a car you could do other things. You're spending a bunch ... it's just another, you know, space where you're spending, uh, a significant part of your life, right? So not having to share it with, uh, other people who you don't want to share it with I think is, uh, uh, a very nice property. Uh, maybe you want to, you know, take a phone call or, you know, do something else in the vehicle. Um, uh, and, you know, uh, safety on the quality of the driving as well as the physical safety of, you know, not having s- you know, to share that ride, uh, is, you know, uh, important to a lot of people.
- 1:11:49 – 1:18:33
Do self-driving cars need to break the rules like humans do?
- DDDmitri Dolgov
- LFLex Fridman
What about the idea that when, when there's a, somebody, like a human driving and they do a rolling stop on a stop sign, like sometimes like, you know, you get an Uber or Lyft or whatever, like human driver and, you know, they can be a little bit aggressive as, as drivers. It feels like there is, um ... not all aggression is bad. Uh, now that may be a wrong, again, 20th century conception of driving; maybe it's possible to create a driving experience ... like if you're in the back busy doing something, maybe aggression is not a good thing. It's a very different kind of experience perhaps. But it feels like in order to navigate this world, you need to, uh ... how do I, uh, phrase this? You need to kind of bend the rules a little bit or at least like test the rules. I don't know what language politicians use to discuss this, but, uh, (laughs) whatever language they use, you like flirt with the rules? I don't know. But like you, uh, you sort of, uh, have a bit of an aggressive way of driving that-... asserts your presence in this world, thereby making other vehicles and people respect your presence, and thereby allowing you to sort of navigate through intersections in a timely fashion. I don't know if any of that made sense, but like how does that fit into the experience of driving autonomously? Is that ?
- DDDmitri Dolgov
Makes a lot of sense. This is, you're hitting on a very important point of, uh, a number of behavioral components and, um, you know, uh, parameters that make your driving feel, you know, assertive and natural, comfortable, predictable. Um, now our cars will follow rules, right? They will do the safest thing possible in all situations, like, you know, be clear on that. Uh, but if you think of really, really, you know, good drivers, just, you know, think about, you know, professional limo drivers, right? They will follow the rules. They're very, very smooth. Uh, and yet they're very efficient. Uh, and but now they're assertive. Uh, they're comfortable for the people in the vehicle. They're predictable for the, uh, other people outside the vehicle that they share the environment with. And that's the kind of driver that we want to build. And you just need to think if, you know, if maybe there's a sport analogy there, right? You know, you can do and, you know, very, in many sports the, uh, true professionals are very efficient in their movements, right? They don't do like, you know, hectic, uh, flailing, right? They're, you know, smooth and precise, right? And they get the best results. So that's the kind of driver that we want to build. In terms of, you know, aggressiveness, yeah, you can like, you know, roll through the stop signs, you can do crazy lane changes. Uh, typically it doesn't get you to your destination faster, typically not the safest or most predictable, uh, or most comfortable thing to do. And, uh, but there is, uh, a way to do both. And that, that, that's what we're doing, we're trying to build a driver that is, uh, safe, comfortable, smooth, and predictable.
- LFLex Fridman
Yeah. Th- that's a really interesting distinction. I think in the early days of autonomous vehicles, the vehicles felt cautious as opposed to efficient and still probably, but when I rode in the Waymo, I mean there was, it was, it was quite assertive. (laughs) It moved pretty quickly. Like, um, yeah, that is one of the surprising feelings was that it actually, it went fast, and it didn't feel like awkwardly cautious that autonomous vehicle. Like, like so I've also programmed autonomous vehicles and everything I've ever built was, felt awkwardly e- either overly aggressive, okay, especially when it was my ca- code, or, uh, like awk- awkwardly cautious is the way I would put it. And Waymo's vehicle felt like, uh, assertive and I think efficient is like, uh, the right terminology here. It wasn't, uh... And I also like the professional limo driver 'cause we often think like, you know, an Uber driver or a bus driver or a taxi... This is the funny thing is, is people think like tr- taxi drivers are professionals. (laughs) They, I mean, it's, it's like, that, that's like saying m- I'm a professional walker just because I've been walking all my life. I think there's an art to it, right? And if you take it seriously as an art form, then the, there's a certain way that mastery looks like. It's interesting to think about what does mastery look like in driving. And perhaps what we associate with like aggressiveness is unnecessary, like it's not part of the experience of driving. It's like unnecessary fluff that, uh, efficiency, you could, you can be... You can create a good driving experience within the rules. That's, uh, I mean you're the first person to tell me this, so it's, it's kind of interesting. I need to think about this. But that's exactly what it felt like with Waymo. I kind of had this intuition, maybe it's the Russian thing, I don't know, that you have to break the rules in life to get anywhere. (laughs) But maybe, maybe it's possible that that's not the case in driving. I have to think about that. But it certainly felt that way on the streets of Phoenix when I was there in, in Waymo, that, that, that was a very pleasant experience and it wasn't frustrating in that like come on, move already kind of feeling. It wasn't, that wasn't there.
- DDDmitri Dolgov
Yeah, I mean, th- that's what, that's what we're going after. I don't think you have to pick one. I think truly good driving, it gives you both efficiency, assertiveness, but also comfort and predictability and, you know, safety. Uh, and, you know, it's, that's what fundamental improvements in the f- core capabilities truly unlock and you can kind of think of it as, you know, precision and recall trade-off. You have certain capabilities of your model and then it's very easy when, you know, you have some curve of precision and recall and you can move things around and can choose your operating point. And you're trading off precision versus recall, false positives versus false negatives, right? But then, and, you know, you can tune things on that curve and be kind of more cautious or more aggressive but then aggressive is bad or, you know, cautious is bad. But true capabilities come from actually moving the whole curve up and, and then you are k- on kind of on a very different plane of those trade-offs. And that, that's what, you know, we're trying to do here is to move the whole curve up.
- 1:18:33 – 1:24:11
Waymo Trucks
- DDDmitri Dolgov
- LFLex Fridman
Before I forget, let's talk about trucks a little bit. Uh, so I also got a chance to check out some of the Waymo tru- uh, trucks. Um, I'm not sure if, uh, we want to go too much into that space, but it's a fascinating one, so maybe we can mention at least briefly. You know, Waymo is also now doing autonomous trucking and, uh, how different like philosophically and technically is that whole space of problems?
Episode duration: 2:23:09
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode P6prRXkI5HM
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome