No Priors Ep. 87 | With Co-CEO of Waymo Dmitri Dolgov

Sarah Guo 0:00 (music plays) Hi, listeners. Welcome to No Priors. Today, we're hanging out with Dmitry Dolgov, co-CEO of Waymo. Waymo started as the Chauffeur Project within Google back in 2009, and eventually

Sarah Guo 0:15 spun off as its own company. Now it provides over 100,000 paid rides each week across San Francisco, LA, Austin, and Phoenix. I love taking Waymos, and I'm regularly campaigning for better South Bay coverage. We're excited to dig

Sarah Guo 0:29 into all things robo-taxis, self-driving, what it takes to deploy this technology on a mass scale, and what's next for Waymo.

Elad Gil 0:36 Hi, Dmitry. Thank you so much for joining us today.

Dmitri Dolgov 0:37 Thank you for having me.

Elad Gil 0:38 Yeah. Uh, maybe we can start off with just a little bit of a history of, um, self-driving at Google, how you got involved, and how things have evolved over time.

Dmitri Dolgov 0:46 I've been do- doing this for, you know, quite a few years. I think, uh, about 18 now. I got started in, uh, around 2006. Uh, this was the time of the DARPA Grand Challenges. Um, this was when, you know, DARPA organized, uh, a few competitions, uh, that they called the Grand Challenge in robotics, uh, uh, for the, uh, with the purpose of advancing research in autonomous vehicles, right? So the first competition they had was the first Grand Challenge. This was, uh, the challenge there was to create a car that could drive autonomously in a desert. I just completed this deck and why I'm gonna, you know, drive for about 100 miles. Uh, nobody succeeded, but, you know, there was a lot of great progress that was made, and then they repeated the challenge, and a few teams succeeded. So on the heels of that, they created another challenge th- called the DARPA Urban Challenge, uh, where the setup was, uh, kind of a mock,

Dmitri Dolgov 1:39 uh, city that was supposed to imitate, you know, what driving on public roads is like. And that's the one, uh, that, uh, I was involved in. I was on Stanford's team. This was kind of my, you know, uh, moment where it clicked for me.

Elad Gil 1:50 Mm-hmm.

Dmitri Dolgov 1:50 I saw the, you know, the future and

Dmitri Dolgov 1:53 the benefits, and I've never looked back. That's what I've been doing ever since. Uh, and then, uh, we started this project at Google in 2009. Uh, it was just me and a small group of us, uh, and then that grew into what now is Waymo, uh, when we started the company, uh, in the very beginning of 2017.

Elad Gil 2:09 All right. Yeah, it seems like a lot of the, um, lineage or history of this field all traces back to a handful of labs. You know, it's like Sebastian Thrun's lab at Stanford and a few others, and it seems like the founders of a lot of the companies that ended up eventually existing in this ecosystem all came out of the same sort of cohort of people, which I always think is fascinating to think about in terms of lineages.

Dmitri Dolgov 2:28 Yeah, yeah. No, definitely a, definitely a small world. The CMU team and the Stanford team, there was, uh, a few, uh, people who came and started this project at Google-

Elad Gil 2:35 Mm-hmm.

Dmitri Dolgov 2:35 ... in, in 2009, uh, came from those teams.

Elad Gil 2:37 Mm-hmm. I think when you started working on this, this was considered a little bit of a crazy thing to do, right? It was early on. A lot of parts of the waves of deep learning hadn't really quite happened yet, um, in terms of o- applications across all sorts of areas, like AlexNet hadn't existed yet, like all these other sorts of things that-

Dmitri Dolgov 2:53 No, nothing existed, right? (laughs)

Elad Gil 2:54 Yeah, yeah, yeah.

Dmitri Dolgov 2:54 Uh, and you're, you're absolutely right. Uh, people, uh, you know, heard a lot about us being crazy and it's, you know, never going to work. In all honesty with ourselves-

Elad Gil 3:04 Mm-hmm.

Dmitri Dolgov 3:04 ... we're not exactly

Dmitri Dolgov 3:05 sure if, you know, we are just a little bit crazy or, you know, completely, yeah, insane, uh, when we're trying to go after this problem.

Sarah Guo 3:11 Did you treat it as an open-ended research project, or you treated it as like how you had an idea in your mind of, like, an end point date where this would be viable on public roads?

Dmitri Dolgov 3:20 More of the former, but it was not, it was not research, right? So no, you know, and then the kind of the university DARPA Challenge date, it was a research project. Uh, yeah. Uh, then when we started Google, it, you know, was u- under the belief that, you know, we can make it work. Uh, and if we can, then the impact, the positive impact of this technology on the world and the mission is, is worth it. Uh, but it was early days, so we actually, you know, uh, had very little data to, you know, go by in terms of, you know, uh, you know, thinking about how long it's gonna take and how hard the problem is going to be.

Elad Gil 3:56 How long did you think it was gonna take? Like, at the time that you started working on this?

Dmitri Dolgov 3:58 Uh ...

Sarah Guo 3:59 In a-

Dmitri Dolgov 3:59 Oh, I, I don't know if we had, you know, a specific date, but I think that was actually the first question that we posed.

Elad Gil 4:03 Mm-hmm.

Dmitri Dolgov 4:03 It's like, let's not build a product, right? In the first, you know, couple of years or so-

Elad Gil 4:06 Mm-hmm.

Dmitri Dolgov 4:06 ... we didn't have, like, a, you know, product in mind or, you know, uh, a target date in mind. The first order of business was to explore the space, right? So we, you know, towards that end, we created some milestones for ourselves, uh, with the goal of prototyping and learning and just understanding. So after those two years, we said, "Ah, okay, you know, there's something there. Let's start talking about, you know, what the product could be." And actually, our first product that we thought was gonna be viable, uh, was, you know, what nowadays you would call kind of an advanced driver

Dmitri Dolgov 4:33 assist system, right? And we had some expectations of, you know, a, a small number of years that it would take for us to get there. Uh, when, you know, we, after working on it for a while and making more progress on kind of the core of the technology, we decided that was not the, you know, right path for us, that we want to go after full autonomy. That was, you know, that made that pivot around 2013.

Elad Gil 4:49 You're

Elad Gil 4:50 now doing something like 100,000 rides a week, so five million rides a year sort of annualized out, which is incredible. What was the inflection point, or what suddenly caused that sort of volume to happen or all these things to come together? 'Cause it feels like a reasonably recent phenomenon in some sense.

Dmitri Dolgov 5:04 Yeah, you're right. Uh, but th- those are discrete, you know, some discrete jumps. So yeah, kind of if, you know, we rewind the clock a little bit, I think in my mind there were a few kind of generational discontinuous steps-

Elad Gil 5:13 Mm-hmm.

Dmitri Dolgov 5:14 ... on kind of that progression from, you know, that point in 2013 when we said let's go for it to where we are today. Right, so exactly right, you know, 100,000 trips per week, more than a million miles per week and growing, you know, exponentially.

Elad Gil 5:25 Mm-hmm.

Dmitri Dolgov 5:25 So some interesting ones were, you know, in, uh, uh, 2015, that was kind of our zero to

Dmitri Dolgov 5:29 one moment. This is when, you know, for the first time, we put a car on the road. Uh, there was that, you know, our, what we call the, you know, third generation, uh, of our system. This was the, uh, third generation of our, you know, self-driving, uh, hardware suite, you know, sensors, computer, and we put it on a custom-designed vehicle that we called the Firefly.

Elad Gil 5:45 Mm-hmm.

Dmitri Dolgov 5:45 Right? We took a few rides with nobody behind the wheel, zero to one moment. Uh, then the next, uh, evolution in that kind of generational step was our fourth generation of our driver. Those are the Pacifica minivans.

Elad Gil 5:57 Mm-hmm.

Dmitri Dolgov 5:57 Um, uh, with the fourth generation of the Waymo hardware suite. And we deployed those, uh, uh, in a full autonomous mode in Arizona i- in Chandler, and we actually opened it up to the public in 2020.Right? Uh, but at that point, uh, we... The focus was on doing it, uh, repeatedly, right? And the focus was on, uh, maturing the, you know, the technology, the building of the driver, the evaluation of the driver-

Sarah Guo 6:24 Mm-hmm.

Dmitri Dolgov 6:24 ... and of doing releases in a regular cadence, getting it out to real customers, hearing from the customers, right? Understanding the feedback, you know, and iterating. So that was the focus of that fourth generation, was not, you know, to grow and scale and capture the market, right? And then at that point, we, uh, made the decision to jump to what, you know, we now call the fifth generation of the Waymo Driver. It's on the JLR, uh, IP. This is what you see in, you know, in the fleet today, in, in those four cities, uh, uh, Phoenix, San Francisco, LA, and you know, Austin.

Sarah Guo 6:52 I thought it was very smart actually, that you started in Arizona, um-

Dmitri Dolgov 6:55 Mm-hmm.

Sarah Guo 6:55 ... versus in California. And I think, you know, for example, um-

Dmitri Dolgov 6:58 Mm-hmm.

Sarah Guo 6:59 ... I, I think crews ran into some issues in San Francisco where there was activists like putting cones on the cars and trying to stop them and doing other things. And so it seems smart to, to start in Arizona. I was just sort of curious what are the criteria that led you to-

Dmitri Dolgov 7:10 Mm-hmm.

Sarah Guo 7:10 ... to start that as a sort of a test bed or a place to-

Dmitri Dolgov 7:13 So I guess, you know, it depends on the different time, you know, horizons. So in, you know, in the fourth, on the fourth generation, we picked, uh, a deployment area that was kind of medium complexity. Uh, and the goal there was, as I mentioned, is to kind of go end to end.

Sarah Guo 7:25 Mm-hmm.

Dmitri Dolgov 7:25 So we picked an environment where we thought it was, you know, we check enough of the boxes to, you know, help us learn the most important things that we wanted to learn and de-risk, right? While... And that was a deployment. Uh, and then there's the development of the system. So for the development of the system, you wanna go after the hardest problems possible, right? You wanna go after the densest environments, you wanna go after the harshest weather.

Sarah Guo 7:42 Mm-hmm.

Dmitri Dolgov 7:42 So we've kind of in parallel been doing that. So we've made a decision to deploy, you know, in Chandler in 2020-

Sarah Guo 7:47 Mm-hmm.

Dmitri Dolgov 7:47 ... you know, to learn from the end-to-end system, while, you know, pushing on developing the system. Then when we, we've, uh, you know, learned enough and we made that discontinuous jump to the fifth generation of our driver, and then we said, "Okay, like that's the platform we believe that we wanna take to scale."

Sarah Guo 8:01 What's the hardest environment for a self-driving car?

Dmitri Dolgov 8:05 So, density matters.

Sarah Guo 8:06 Mm-hmm.

Dmitri Dolgov 8:06 Speed matters.

Sarah Guo 8:07 Yeah.

Dmitri Dolgov 8:07 Uh, weather matters.

Sarah Guo 8:08 Okay.

Dmitri Dolgov 8:09 So kind of, you know, where, you know, those come together-

Sarah Guo 8:10 Mm-hmm.

Dmitri Dolgov 8:11 ... uh, is, uh, where most complexity is, right?

Sarah Guo 8:13 Mm-hmm. So New York in the winter is really bad. And then-

Dmitri Dolgov 8:16 Yeah.

Sarah Guo 8:17 ... you know.

Dmitri Dolgov 8:17 That's right.

Sarah Guo 8:18 Yeah.

Dmitri Dolgov 8:18 That's right. Uh, but then this is why we picked San Francisco is that, you know, it's good for learning, uh, advancing the driver. It is a very interesting, you know, commercial market. Uh, we also in the same time went, uh, after downtown Phoenix. It has, you know, a different mix, more, more of the higher speed roads. And, uh, that gives us, you know, kind of, uh, the way we think about it at terms of the development and evaluation of the driver is kind of in the space of the operating domain, not necessarily areas or zip codes. And then you kind of take cities and you map that to the operating domain and, and you deploy, right?

Sarah Guo 8:46 Hm.

Dmitri Dolgov 8:46 And then looking forward is the kind of in terms of how we think about, you know, future cities. It is, you know, the, a few lenses. It is market.

Sarah Guo 8:52 Mm-hmm.

Dmitri Dolgov 8:53 Like, you know, is, is there a good, you know, uh, market from the commercial perspective? Uh, what is the technical, uh, complexity, you know, what is the, you know, regulatory environment? Uh, and so-

Sarah Guo 9:02 Hm.

Dmitri Dolgov 9:02 ... and that's how kind of the lens that we, uh, apply.

Sarah Guo 9:05 Mm-hmm. Yeah. What were the big technology breakthroughs that got you to this fifth generation driver that is really the one that you think you can scale now?

Dmitri Dolgov 9:11 Uh, so the biggest one's in AI.

Sarah Guo 9:12 Mm-hmm.

Dmitri Dolgov 9:12 Probably, you know, not, not surprising. Uh, you know, there was generational breakthroughs and, you know, with every generation of the hardware of our, you know, of the driver, there's, you know, new hardware. So it's getting more capable, it's getting simpler, it's getting, you know, uh, you know, more inexpensive and there's, you know, a lot of simplification. But that's,

Dmitri Dolgov 9:26 that's, that's a boost, right? Really, it's all about, you know, AI as you mentioned.

Sarah Guo 9:29 And what, what specifically in AI was the, the shift or change that was important? Was it moving to sort of end-to-end DL for everything? Was it the transformer backbone? Was... I'm just sort of curious, like was there a specific thing that-

Dmitri Dolgov 9:39 So for that last jump, the models, you know, transformers, as you mentioned-

Sarah Guo 9:42 Mm-hmm.

Dmitri Dolgov 9:42 ... played a huge role, right?

Sarah Guo 9:43 Hm.

Dmitri Dolgov 9:43 Before that, you know, we had the big breakthrough before that, you know, it was, uh, ConvNets, you mentioned AlexNet that was-

Sarah Guo 9:48 Mm-hmm.

Dmitri Dolgov 9:48 ... around 2013.

Sarah Guo 9:49 Yeah.

Dmitri Dolgov 9:49 So that gave us a big boost, but was still kind of a plateaued at kind of, you know, the wrong place.

Sarah Guo 9:54 Hm.

Dmitri Dolgov 9:54 And then it was a few of those things coming together, right?

Sarah Guo 9:56 Hm.

Dmitri Dolgov 9:56 It is, you know, transformers, uh, it is, you know, bigger models, more compute coupled with kind of the whole, you know, evaluation measures. We often talk about the architectures and what's really m- more important kind of the machine surrounding the architecture. You need to ask, you know, the architecture is an enabler-

Sarah Guo 10:09 Mm-hmm.

Dmitri Dolgov 10:10 ... but really to make it work at the level that we care about, you need like everything around it, the data engine that will, you know, flywheel of, you know, training the system, evaluating it. And you kind of have to think about the problem of evaluating the driver in tandem of building it, right? And, uh, you know, you need to simulate it, you need data. So all of those coming together I think is what leads to-

Sarah Guo 10:28 Mm-hmm.

Dmitri Dolgov 10:28 ... uh, you know, the, the breakthrough and discontinuity that you're seeing-

Sarah Guo 10:31 Mm-hmm.

Dmitri Dolgov 10:31 ... with, uh, where we are today.

Sarah Guo 10:32 Yeah. Can you explain how you guys think about evaluation internally and then also, um, you know, how that might differ from how regulators evaluate this from the safety case perspective?

Dmitri Dolgov 10:42 So the in- that's a big question, uh, but in, uh, I think, I'm glad you're asking about because it is a super important and an like insanely difficult problem, right? We often talk again about the building of the driver, but there's two problems. The building of the driver and the evaluation of it. And they go, you know, hand in hand. So, you know, internally it starts with,

Dmitri Dolgov 11:01 you know, figuring out what metrics you care about, then, uh, bringing the data to support the evaluation of those metrics, you know, which we, you know, have hundreds.

Sarah Guo 11:09 Hm.

Dmitri Dolgov 11:09 Uh, then you need all the infrastructure, uh, including things like the simulation. There are some things you can evaluate in open loop, you know, kind of, and some things you need closed loop simulation for. So you need to build, you know, a s- a realistic scalable simulator to support all of that, right?

Sarah Guo 11:23 Mm-hmm.

Dmitri Dolgov 11:23 There are, you know, all of these metrics that guide our, you know, development of the system that, you know, help us, uh, improve and kind of train, uh, the, the Waymo AI, the Waymo driver, and then it funnels into what we call validation and evaluation. The aggregate of the evaluation and validation methodology is what we call the readiness and safety framework.

Sarah Guo 11:41 You know, if you look at-

Elad Gil 11:42 ... uh, miles driven in an urban setting, or-

Dmitri Dolgov 11:45 Mm-hmm.

Elad Gil 11:45 ... you know, some comparison to human drivers in a given city that you operate in-

Dmitri Dolgov 11:48 Yeah.

Elad Gil 11:49 ... what is the relative safety level of what Waymo's doing versus a human driver at this point?

Dmitri Dolgov 11:52 We are pretty proud, uh, of our record. I think we have now, now that we've driven, you know, tens of millions, uh, of miles in fully autonomous, what we call rider only mode-

Elad Gil 12:01 Mm-hmm.

Dmitri Dolgov 12:01 ... uh, and driving, you know, today more than a million miles, uh, per week-

Elad Gil 12:05 Mm-hmm.

Dmitri Dolgov 12:05 ... I think we can, you know, with pretty good confidence in empirical data say that we are, you know, better than human's benchmarks.

Elad Gil 12:10 Mm-hmm.

Dmitri Dolgov 12:10 So we published some of that, you know, very recently, uh, on what we call the Safety Hub.

Elad Gil 12:13 Mm-hmm.

Dmitri Dolgov 12:14 Uh, the latest data point that we shared was based on 22-

Elad Gil 12:17 Mm-hmm.

Dmitri Dolgov 12:17 ... uh, million fully autonomous rider only miles.

Elad Gil 12:19 Mm-hmm.

Dmitri Dolgov 12:19 And, uh, we compared our performance-

Elad Gil 12:23 Mm-hmm.

Dmitri Dolgov 12:23 ... uh, versus human benchmarks, uh, by, you know, different severity levels of, you know, context of collisions, different severity levels. So we see, depending on the severity level, we see that, you know, for the lowest severity, lower severity outcomes were about, um, you know, uh, a factor of two better-

Elad Gil 12:38 Mm-hmm.

Dmitri Dolgov 12:38 ... than the human's benchmarks.

Elad Gil 12:39 Mm-hmm.

Dmitri Dolgov 12:39 And as you look at more severe outcomes-

Elad Gil 12:42 Mm-hmm.

Dmitri Dolgov 12:42 ... the gap increases. So for kind of the most severe-

Sarah Guo 12:45 That's pretty good.

Dmitri Dolgov 12:46 ... that's exactly what we wanna see, right?

Sarah Guo 12:47 Mm-hmm.

Dmitri Dolgov 12:47 So for airbag, you know, deployment type collisions, we're about a factor of six, uh, better than human drivers.

Elad Gil 12:52 Mm-hmm. Yeah.

Dmitri Dolgov 12:52 And that's, uh, without any notion or attribution of causality or faults.

Elad Gil 12:57 Right.

Dmitri Dolgov 12:57 If you bring that into the picture, you know, most of those are kind of-

Elad Gil 13:00 Sure.

Dmitri Dolgov 13:00 ... unavoidable. There's very little they can do.

Sarah Guo 13:02 Yeah.

Dmitri Dolgov 13:02 We have done a study with, um, uh, Swiss Re, the global, uh, kind of, I think, the biggest global range of insurance in the world.

Sarah Guo 13:08 Sure. Yeah.

Dmitri Dolgov 13:08 Uh, and we partnered with them. We shared the data. They've done the analysis, and they found that, you know, for damage claims, we had about a 4X reduction versus-

Elad Gil 13:16 Mm-hmm.

Dmitri Dolgov 13:16 ... the human baseline, and for bodily injury claims-

Elad Gil 13:19 Mm-hmm.

Dmitri Dolgov 13:19 ... we had 100% reduction. It was a smaller data set. This was, you know, e- earlier, but that was based on just a little bit-

Sarah Guo 13:24 Mm-hmm.

Dmitri Dolgov 13:24 ... less than four million miles, but also, you know, uh, uh, it was, from their point of view, statistically significant, uh, and, um, that, that, you know, again, we are, we feel pretty proud of that.

Sarah Guo 13:34 Yeah. It's amazing. Um, given that, what do you think the regulatory stance should be?

Dmitri Dolgov 13:38 I think we wanna make sure that, you know, we enable, uh, this, uh, technology and service of the mission of making roads safer-

Elad Gil 13:47 Mm-hmm.

Dmitri Dolgov 13:47 ... uh, and, you know, we've been engaged with regulators, you know, for many years and have that dialogue and, you know, so far, we've, uh, had good success, uh, getting all the necessary, you know, permits and all of, uh, to allow to scale. The way we think about it, uh, internally, and that's how kind of we have the dialogues with the regulators, with communities, with riders, it needs to be based on transparency, and it needs to be in responsible, iterative, gradual process, right? Because this thing is very new. The technology is very new. The products is, you know, very different.

Elad Gil 14:15 If you're doing a million miles a week now, what prevents faster rollout? In other words, it, it seems like you've proven now that this is a safe solution.

Dmitri Dolgov 14:22 Mm-hmm.

Elad Gil 14:22 It's working very well at scale. You have your generation five driver that seems to be quite performant.

Dmitri Dolgov 14:27 Yeah.

Elad Gil 14:28 Why not go bigger, faster?

Dmitri Dolgov 14:30 We are scaling, you know, exponentially.

Elad Gil 14:33 Mm-hmm.

Dmitri Dolgov 14:33 Uh, it took us, uh, about three months to get from 50,000-

Elad Gil 14:37 Mm-hmm.

Dmitri Dolgov 14:37 ... miles to 100,000 miles.

Elad Gil 14:39 Mm-hmm.

Dmitri Dolgov 14:39 Right? So, you know, we're moving-

Elad Gil 14:40 Sure.

Dmitri Dolgov 14:41 ... at a, at a, at a good rate. Uh, but the main thing, uh, is-

Elad Gil 14:44 What is the total number of miles that are driven in the US per year?

Dmitri Dolgov 14:47 Uh-

Elad Gil 14:47 I'm sorry to put this on a fractional basis.

Dmitri Dolgov 14:49 All, all vehicles, I think it's, uh, you know, in different modalities-

Elad Gil 14:51 Mm-hmm.

Dmitri Dolgov 14:51 ... you know, large vehicles, small vehicles, I think it's, uh, just over three trillion miles.

Elad Gil 14:54 Okay.

Dmitri Dolgov 14:55 So quite a bit.

Elad Gil 14:55 Yeah.

Dmitri Dolgov 14:55 We ha- we ha- we ha- we have-

Elad Gil 14:56 (laughs) We have that on our way.

Dmitri Dolgov 14:58 (laughs) But the way, again, you know, the way we think about it is that it's important for this to be kind of an iterative process where we earn trust.

Elad Gil 15:06 Mm-hmm.

Dmitri Dolgov 15:07 It is not a thing where, you know, you-

Elad Gil 15:08 Mm-hmm.

Dmitri Dolgov 15:08 ... you- you know, build it and then you, uh-

Elad Gil 15:10 Mm-hmm.

Dmitri Dolgov 15:10 ... you know, just turn on the switch and-

Elad Gil 15:12 Sure.

Dmitri Dolgov 15:12 ... that's great, right? It needs to be something new, different. There needs to be a dialogue. It needs to be like we, you know, talked about establishing, uh, the safety-

Elad Gil 15:19 Sure.

Dmitri Dolgov 15:19 ... record, right? And we needed to build up to that, right?

Elad Gil 15:21 Mm-hmm.

Dmitri Dolgov 15:21 We collected in tens of millions miles. Now, we feel pretty good about that. So then we, you know, that gives us confidence-

Elad Gil 15:26 Mm-hmm.

Dmitri Dolgov 15:26 ... that earns us, you know, trust.

Elad Gil 15:28 Yeah.

Dmitri Dolgov 15:28 Uh, so you have to be transparent about where we are, and then we, you know, uh-

Elad Gil 15:31 What do you think-

Dmitri Dolgov 15:32 ... grow and scale.

Elad Gil 15:32 ... is, uh, the biggest shift that's coming from a technology basis going forward? We, we had, um-

Dmitri Dolgov 15:36 Mm-hmm.

Elad Gil 15:36 ... André Comprathie on our podcast a couple weeks ago, and he sort of contrasted what he viewed as the Tesla approach, which is more, in his view, sort of software driven, to the, um, Waymo approach, which was more sort of hardware centric. Um, do you think that's a correct characterization? And also, how do you view things sort of changing over the next, you know, uh, couple years or year or two in terms of your roadmap?

Dmitri Dolgov 15:57 Yeah. Uh, I would think, yeah, I mean, it's all about AI.

Elad Gil 15:59 Mm-hmm.

Dmitri Dolgov 16:00 With a full stop. Like, we talked about, you know, a few, uh, big breakthroughs that allowed us to get to where we are today-

Elad Gil 16:05 Mm-hmm.

Dmitri Dolgov 16:05 ... you know, continets, transformers, bigness now, you know, most recently kind of combining, uh, the Waymo-

Elad Gil 16:10 Mm-hmm.

Dmitri Dolgov 16:11 ... uh, AI, with the congenial knowledge of, you know, VLMs, thi- this is at the core of it. Uh, and, you know, hardware matters, right? We all have to operate in the physical world.

Elad Gil 16:20 Mm-hmm.

Dmitri Dolgov 16:20 Uh, so the hardware that we have, um, on our cars gives us, I tend to think of it as an advantage, right? You ha- like, you have to-

Elad Gil 16:26 Mm-hmm.

Dmitri Dolgov 16:26 ... see well.

Elad Gil 16:27 Yeah.

Dmitri Dolgov 16:27 Right? Uh, so, you know, as a human driver-

Elad Gil 16:30 Mm-hmm.

Dmitri Dolgov 16:30 ... right, like if you can't see, you know, uh, if you close one eye-

Elad Gil 16:34 Mm-hmm.

Dmitri Dolgov 16:34 ... you don't, you know, you lose depth. If you, like if your-

Elad Gil 16:37 Sure.

Dmitri Dolgov 16:37 ... you know, your vision is not 20/20, you don't have your glasses, like you'll still drive, right? You'll drive okay-

Elad Gil 16:41 Mm-hmm.

Dmitri Dolgov 16:41 ... but, you know, you're not gonna be as good of a driver.

Elad Gil 16:43 Mm-hmm.

Dmitri Dolgov 16:43 But the point, I guess the, the, the way I see it is it's all about AI. It's all about building-

Elad Gil 16:48 Mm-hmm.

Dmitri Dolgov 16:48 ... the system, building the AI, building the software-

Elad Gil 16:50 Mm-hmm.

Dmitri Dolgov 16:51 ... and being able to evaluate.

Dmitri Dolgov 16:52 And that's what we have today, right?

Elad Gil 16:53 Mm-hmm.

Dmitri Dolgov 16:53 So far, we, we have, you know, achieved good quality-

Elad Gil 16:56 Sure.

Dmitri Dolgov 16:56 ... of the driver. We have built all the machinery to evaluate it and know-

Elad Gil 16:59 Mm-hmm.

Dmitri Dolgov 16:59 ... what it takes. So now for us, it's an optimization.

Elad Gil 17:02 Mm-hmm.

Dmitri Dolgov 17:02 Right? It's a, it's an optimization, simplification, and if we have that, you know, that, uh, the thing that works, and you have a good mechanism-

Elad Gil 17:08 Mm-hmm.

Dmitri Dolgov 17:09 ... to evaluate it, then, you know, it, it really is drastically different-

Elad Gil 17:13 Mm-hmm.

Dmitri Dolgov 17:13 ... in terms of, you know, how fast you can move.

Elad Gil 17:15 Yeah.

Dmitri Dolgov 17:15 And I guess I've been in, in this other mode for years-

Elad Gil 17:17 Mm-hmm.

Dmitri Dolgov 17:18 ... where we have not solved the kind of the, haven't cracked the nut, right?

Sarah Guo 17:20 Yeah.

Dmitri Dolgov 17:20 And you're kind of hypothesizing what it would take and, like, you know, yeah, what is the yield of this technical breakthrough? And you're kind of climbing uphill, right? And this is, you know, old analogy of, you know, going to mountain. You see the peak.-

Sarah Guo 17:30 Mm-hmm.

Dmitri Dolgov 17:30 ... Then you get there, and you thought it was the summit, but no.

Sarah Guo 17:32 Mm-hmm.

Dmitri Dolgov 17:32 Now you see like the, you know, the, the landscape. And that's what it felt like for us, you know, for many years. That's kind of the trend that the in- industry usually follows.

Elad Gil 17:39 Mm-hmm.

Dmitri Dolgov 17:39 And I just find it that it's a qualitatively different place to be in when you've cracked the nut-

Elad Gil 17:43 Mm-hmm.

Dmitri Dolgov 17:43 ... and you have the evaluation, and now you can optimize and scale.

Sarah Guo 17:45 As part of your optimization, um, do you think of, like, reducing or simplifying the sensor suite as an important priority?

Dmitri Dolgov 17:52 Every generation of the hardware, you know, it, uh, increase capability, but also simplify-... drastically, and then, uh, the cost comes down-

Elad Gil 17:59 Mm-hmm.

Dmitri Dolgov 17:59 ... uh, every generation, so that was true, you know, for the previous generations. Uh, we made a big, uh, jump from the, uh, you know, the p- previous generation to today's, and then, uh, from the fourth to the fifth, and then now going to- to the sixth generation. This was the primary focus, simplification and bringing down the cost, as well as, you know, with the vehicle, making it about the user experience. It's absolutely ... And then of course, the economies of scale, right? There's nothing like fundamentally, look at the components that we use-

Elad Gil 18:23 Mm-hmm.

Dmitri Dolgov 18:23 ... uh, you know, in our hardware. There's just, you know, all of them are, you know, fairly, you know, commoditized and with scale, they just, you know, they, you think, get- get to ride the typical, you know, uh, curves of the hardware. You know, radars used to be expensive in the past, right? You put them all in the cars, you brilliantly bring down the cost. You know, same with computers, right?

Elad Gil 18:39 I guess qualitatively, we know that, uh, people can drive well with like a very simple vision system, right?

Dmitri Dolgov 18:46 Yeah.

Elad Gil 18:46 And obviously in the self-driving world, we have dramatically more and different types of sensors. Is there any sort of analytical framework that you all use in terms of the amount of data/types of data you need to collect in order to have a performance system relative to the curve on the-

Dmitri Dolgov 18:59 Mm-hmm.

Elad Gil 18:59 ... AI side? In other words-

Dmitri Dolgov 19:01 Yeah.

Elad Gil 19:01 ... can you, can you follow some sort of scaling curve on AI that you can pre-predict the set of sensors that you can do without?

Dmitri Dolgov 19:08 What we have now, uh, is, you know, now that, kind of to jump to the- the core of the answer to your question, now that we have, you know, cracked the nut of the- the driver and the evaluation, we can answer that question with data.

Elad Gil 19:20 Mm-hmm.

Dmitri Dolgov 19:20 Right? So we have, you know, talked about different stuff, you see, uh, start with humans, we have, you know, just two kind of, you know, uh, cameras, you know, but like on a- on a- on a- on a pivot, right?

Elad Gil 19:27 Mm-hmm.

Dmitri Dolgov 19:28 Uh, you know, we can drive kind of okay-ish, right?

Elad Gil 19:29 Yeah.

Dmitri Dolgov 19:30 Uh, and then, you know, if your vision is not very good or if your, you know, engine, like you maybe not, you know, drive as well as computers can, right? Um, so that's why we have vision, we have, uh, cameras, we have LiDARs, we have radars, right? And they all give us, you know, some benefits. Uh, and we can talk about, you know, what the pros and, uh, of that they all and kind of how they, uh, they bring to the table, the data they bring to the table and how they're nicely complementary, right, and what you get from the different sensing modalities just by the kind of the physics of it. But to, again, in terms of, you know, how do we, you know, answer that question, uh, you know, for years it was more theoretical and you hypothesized, you know, how much of, you know, a certain thing you need.

Elad Gil 20:04 Mm-hmm.

Dmitri Dolgov 20:04 Now that we have a thing that works and we can evaluate it, we can, you know, bring data to the table.

Elad Gil 20:08 Mm-hmm.

Dmitri Dolgov 20:08 We can empirically, more empirically, you know, answer that question and say what- what happens to the Waymo driver if you take away something, right?

Elad Gil 20:15 Mm-hmm.

Dmitri Dolgov 20:15 Maybe you add noise to the system, maybe you degrade, maybe you take away a full sensor modality. Maybe you take away LiDAR and you take radar away completely and you just drive with camera. You know, will this drive? Yeah. You know-

Elad Gil 20:24 Mm-hmm.

Dmitri Dolgov 20:24 ... you can use a human driver with one eye or, you know, with, uh, blurry vision, yeah. Are you, is that acceptable performance? Will you get a license? No.

Elad Gil 20:31 Mm-hmm.

Dmitri Dolgov 20:31 Same thing for us. We can, you know, take stuff away and we can answer that question, like is the performance ... Can it still drive? Of course. Is the performance good enough, uh, for full autonomy and is it good enough for our bar of readiness and safety? And the answer is no.

Elad Gil 20:44 Mm-hmm.

Dmitri Dolgov 20:45 Right? Uh, and, but again, this is in the context of full autonomy, it is in the, in the context, uh, of, uh, uh, scale.

Elad Gil 20:52 Mm-hmm.

Dmitri Dolgov 20:53 And it is in the context of the, you know, responsible deployment and the high bar for safety and readiness that we set for ourselves. If you change, you know, some of those inputs, let's say you talk about, you know, small scale or you talk about, you know, not full autonomy, you talk about a driver assist system, then the answer changes, right? You might have a different ... You know, if you still have a human in the loop and they're responsible for safety, you know, a- a different configuration, you know, whether it's, you know, just you get rid of sensor modalities altogether or in, you know, maybe use all three modalities, but you pick a different operating points because you know, it's cameras, it's cameras, right? The resolution, dynamic range, cleaning, you know, same for LiDARs, but, you know, the answer of your operating point would change based on kind of where you set the bar and what the product actually is.

Elad Gil 21:30 If you look forward a year or two, like now, you know, crack the nut, Waymo is looking at scale and, um, and probably more about the business, like do you think of robo-taxis at scale as the- the near term business plan? Or are there other modalities or like deployment? Like if I think about this as just like a CapEx problem-

Dmitri Dolgov 21:47 Mm-hmm.

Elad Gil 21:47 ... like other avenues that are important for you guys to explore.

Dmitri Dolgov 21:50 That's the main one. Ride hailing is the main one that we're focusing on. So we are, you know, focused on technology, we're focused on the product, we are learning from our users, we're, yeah, every day we're earning trust, uh, and we're setting up the ecosystem of, you know, partnerships in that space. So that's our primary focus, right? We are very excited about, uh, the commercial opportunity there. Uh, it's not the only one. Always, you know, think of Waymo as, you know, a technology company who are building a generalizable Waymo driver, right?

Elad Gil 22:16 Okay.

Dmitri Dolgov 22:16 With the mission to build the world's most trusted driver. And we want to deploy that driver not just in ride hailing, right? There's, as said, there's more than three trillion miles in the US.

Elad Gil 22:26 Yeah.

Dmitri Dolgov 22:26 There's more than 10, you know, uh, trillion miles driven, you know, worldwide. Uh, so the vision and the mission is to deploy the Waymo driver in, you know, different commercial products, uh, in different applications and maybe in different, you know, modalities across, uh-

Elad Gil 22:38 Mm-hmm.

Dmitri Dolgov 22:38 ... all of that spectrum. So that includes, uh, you know, things like, uh, deliveries, there's things like, you know, long haul trucking, it includes, uh, you know, things like personally owned vehicles. Uh, but right now we're very focused on ride hailing.

Elad Gil 22:49 Do you think you'd license that out or do you think you'd actually build the vehicles for these different use cases?

Dmitri Dolgov 22:54 Uh, we're not, you know, uh, we've never been, uh, uh, in the business of building vehicles.

Elad Gil 22:59 Mm-hmm.

Dmitri Dolgov 22:59 Uh, we partner, right? And I guess this is how-

Elad Gil 23:02 Mm-hmm.

Dmitri Dolgov 23:02 ... like this is ... The mission is so important.

Elad Gil 23:04 Mm-hmm.

Dmitri Dolgov 23:04 The opportunity is so massive that we don't want to go it alone. So we think about, you know, partnering and weaving together the e- ecosystem to pursue, you know, all of those different commercial, uh, applications and all of those-

Elad Gil 23:16 Mm-hmm.

Dmitri Dolgov 23:16 ... you know, products. And you can see us, uh, you know, doing that in the ride

Dmitri Dolgov 23:19 hailing business, right? We don't build our own vehicles, right? We partner, we partner with-

Elad Gil 23:22 Mm-hmm.

Dmitri Dolgov 23:22 ... Tier One's, we partner with Williams, we partner, you know, with, uh-

Elad Gil 23:25 Mm-hmm.

Dmitri Dolgov 23:25 ... other companies who help us on our operational side, you know, we partner with companies-

Elad Gil 23:28 Mm-hmm.

Dmitri Dolgov 23:28 ... who help us on the network side, so forth and so on.

Elad Gil 23:30 How do you think car ownership will change over the next decade or two? So if, uh, you know, it's- it's very exciting to see this ramp in terms of, you know, driverless rides that are happening.

Dmitri Dolgov 23:39 Mm-hmm.

Elad Gil 23:39 And one could argue at some point some proportion of the population, just like people flip to Uber to, um, you know, ferry them around in major cities versus driving or taking taxis or other things, there could be a flip here to- to autonomous systems versus owning cars, right? You should just be able to order something on demand, have it show up at the right time and you just get in and it takes you wherever you need to go.

Sarah Guo 23:59 Do you view that as a 10% use case, a 30%? I'm just sort of curious-

Dmitri Dolgov 24:03 Yeah.

Sarah Guo 24:03 ... like, what proportion of miles do you think will convert over time?

Dmitri Dolgov 24:06 Uh, I think over time, we'll see more and more as, you know, as, uh, technology matures, as, you know, it gets deployed-

Sarah Guo 24:11 Mm-hmm.

Dmitri Dolgov 24:11 ... in more of these, uh, different products and modalities.

Sarah Guo 24:14 Yeah.

Dmitri Dolgov 24:14 I think you're starting to see, uh, some of that, uh, even today in ride hailing, right?

Sarah Guo 24:17 Mm-hmm.

Dmitri Dolgov 24:17 And it's not uniform, right? But even in kind of the densest cities, if you look at, you know, the people who live in San Francisco or, you know-

Sarah Guo 24:23 Sure.

Dmitri Dolgov 24:23 ... people who live in New York, even, uh, before, you know, autonomy, right, uh, there was a shift of, you know, fewer people wanted to... in those areas, wanted to, uh, own cars, especially the younger generations, right? It's, uh, uh, not- not what they're excited about, right? Mm-hmm. Uh, and I think with, you know, with autonomy in those areas, in kind of the densest cores, you will see, you know, more of that evolution and continued trend, right? And then- then, you know, over time, it will expand, and so on, and so on.

Sarah Guo 24:45 Yeah. Right.

Dmitri Dolgov 24:45 But, you know, the thing that I'm most excited about is in all of those modalities, we'll be bringing the safety benefits-

Sarah Guo 24:49 Mm-hmm.

Dmitri Dolgov 24:49 ... of this technology, uh, to the ecosystem.

Sarah Guo 24:51 One of the reasons I ask is I remember, um, talking with people in the self-driving world, I don't know how long ago, eight years ago, nine years ago, whatever it was. And at the time, everybody thought this wave was coming. I think a lot of people were off in terms of the timeframe-

Dmitri Dolgov 25:03 Yeah.

Sarah Guo 25:03 ... in terms of what actually happened, and there was sort of a flurry of startups all getting up and running at that time. And a lot of the conversations were around how urban environments would change over time.

Dmitri Dolgov 25:12 Mm-hmm.

Sarah Guo 25:12 Where do you actually put a parking lot or where do you park your car versus having, like, a lot outside of the city that would then c-... the cars would come in and pick you up at the right moment versus everything need to be centralized? And so one of the reasons I was asking that question was to try and get a sense of your view of the order of magnitude and also timeframe by which some of these transformations may occur.

Dmitri Dolgov 25:29 You know, I don't want to speculate on specific timeframes. I think the vision, uh, is absolutely the correct one, right?

Sarah Guo 25:34 Mm-hmm.

Dmitri Dolgov 25:34 And so, you know, we talked about the safety benefits of it. That's the primary one. Then there's accessibility. Uh, once you have... You know, you get those benefits once you can deploy at scale, right?

Sarah Guo 25:42 Mm-hmm.

Dmitri Dolgov 25:42 And once you are deployed at scale, you can do things like, uh, you know, use land in a better way, right?

Sarah Guo 25:49 Mm-hmm.

Dmitri Dolgov 25:49 Instead of, you know, taking up so much space for parking lots or having, you know, personally owned vehicles that, you know, just sit around 90% of the time-

Sarah Guo 25:57 Mm-hmm.

Dmitri Dolgov 25:57 ... you can do better. I think as a society, we can do better, right?

Sarah Guo 26:00 Mm-hmm.

Dmitri Dolgov 26:00 Uh, all of the, um... But, you know, the thing with those, uh, benefits is that come from scale, right?

Sarah Guo 26:04 Mm-hmm.

Dmitri Dolgov 26:04 And, uh, again, safety

Dmitri Dolgov 26:06 is the primary one that we're very focused on. You know, we've talked about as kind of the North Star, as the mission, as the vision for, you know, many years, uh-

Sarah Guo 26:14 Mm-hmm.

Dmitri Dolgov 26:14 ... but, you know, it was always kind of with this caveat, once we get to scale. So I think today, we're starting to actually earn the right to talk about, you know, starting to realize that mission, right?

Sarah Guo 26:24 Mm-hmm.

Dmitri Dolgov 26:24 You know, uh, tens of millions of miles behind umbrella, but, you know, more than a million per week.

Sarah Guo 26:29 Mm-hmm.

Dmitri Dolgov 26:30 We... And the benchmarks, the safety benchmarks that are, you know, statistically significant to seem, you know, empirically unambiguous-

Sarah Guo 26:35 Mm-hmm.

Dmitri Dolgov 26:35 ... can talk about actually, you know, real, tangible safety benefits and,

Dmitri Dolgov 26:40 like, inducing- reducing, you know, harm and injuries that are happening on the road today. So that's our primary focus, and as, you know, I think that's the primary, uh, benefit that we'll see. And then beyond that, I think there's going to be auxiliary ones like the ones we mentioned now.

Sarah Guo 26:51 What do you think is the role of, uh, traditional OEMs in- in this world, um, uh, when, you know, I'd- I'd say functionally, like, a car takes you from point to point, ride hailing takes you from point to point with like, you know, different tiers of comfort level. But, uh, you know, very large industry has been built around people buying passenger vehicles for, like, you know, industrial design or brand or all these other things, um, that, uh- uh, you know, you really need a Ford Raptor to run around the Bay Area, right? Um, well, how do you- how do you think that evolves-

Dmitri Dolgov 27:24 Uh...

Sarah Guo 27:25 ... when- when, you know, perhaps one of the more primary, um, drivers of value is now AI and the ability to do this autonomously?

Dmitri Dolgov 27:32 Right. But I guess, you know, we think of what we're doing as building the driver.

Sarah Guo 27:36 Mm-hmm.

Dmitri Dolgov 27:37 And...

Sarah Guo 27:37 And the driver could still drive different cars.

Dmitri Dolgov 27:38 And the driver... Exactly.

Sarah Guo 27:39 Mm-hmm.

Dmitri Dolgov 27:40 You put the driver, but you still need the car.

Sarah Guo 27:42 Mm-hmm.

Dmitri Dolgov 27:42 And different form factors, you know, whether it's, uh, a car that's good for ride hailing in certain, you know, urban environments, whether it's, you know, a- a different vehicle that you need for good transport or truck or, you know, something that you want to take on longer trips with, you know, family.

Sarah Guo 27:54 Mm-hmm.

Dmitri Dolgov 27:54 You know, we will need different cars. We'll need different form factors. And, uh, I think it's very, very complementary-

Sarah Guo 27:59 Mm-hmm.

Dmitri Dolgov 27:59 ... what we're doing and what, uh, you know, the, uh, the car industry, uh, is building.

Sarah Guo 28:05 Perhaps to Elad's point about how cities will change, the answer has clearly been given the efforts to change both drivers and cars and the environment to just change the driver, right? Um, which is what- what you guys have done. Do you think there are arguments still to change the infrastructure, right? Like, you can make, for example, in the public transport space, right, there's other form factors that require the participation of the public sector in order to deploy.

Dmitri Dolgov 28:31 Uh, absolutely. Sustainability is- is, you know, very important for us. Safety is the primary thing, but, uh, I think all of those modalities can coexist. Uh, in fact, uh, just, uh, in the last couple of days, we, uh, announced,

Dmitri Dolgov 28:43 um, uh, something that we're doing where we are incentivizing people to take Waymos in, uh, the cities where we operate to public transit hubs.

Sarah Guo 28:51 Mm-hmm.

Dmitri Dolgov 28:51 And then, you know, everybody benefits.

Sarah Guo 28:53 Sure. How do you think about the form factor of the car itself? I know that there was companies like Zoox that Amazon bought where-

Dmitri Dolgov 28:58 Mm-hmm.

Sarah Guo 28:58 ... they kind of hollowed out the inside of the car because you no longer needed the steering column and everything else.

Dmitri Dolgov 29:02 Mm-hmm.

Sarah Guo 29:02 And they put seats facing each other, almost like a London cab.

Dmitri Dolgov 29:05 Mm-hmm.

Sarah Guo 29:06 Uh, do you have any thoughts on what that experience will look like in the future as more and more things move to autonomous self-driving ride hailing systems?

Dmitri Dolgov 29:14 Yeah. So designing a car, uh, around the passengers makes total sense.

Sarah Guo 29:19 Mm-hmm.

Dmitri Dolgov 29:19 Right? Like, we, you know, uh, in the past, we've designed cars around, you know, primarily the- the driver.

Sarah Guo 29:24 Mm-hmm.

Dmitri Dolgov 29:24 Right? Uh, if, you know, it's the Waymo driver, it's all about the- the rider experience.

Sarah Guo 29:29 Mm-hmm.

Dmitri Dolgov 29:29 Right? So we have, uh, done, you know, quite a bit of work on the sixth generation of the, you know, the Waymo driver and the car. And the car is designed, uh, around with- with the passenger in mind, right?

Sarah Guo 29:41 Mm-hmm.

Dmitri Dolgov 29:41 So it is more spacious. Uh, it is all about the user experience. You know, you have, you know, flat floors. You have, you know, lower, uh, floor for entry. You have doors that slide to the side. So it's all about, you know, getting in, so absolutely.

Sarah Guo 29:53 Mm-hmm.

Dmitri Dolgov 29:53 You know, there's different aspects of it, like, you know, we don't have cars facing each other.

Sarah Guo 29:56 Mm-hmm.

Dmitri Dolgov 29:56 I think there's... You know, it's an open question. Like, some people get, uh, you know-

Sarah Guo 29:59 Mm-hmm.

Dmitri Dolgov 30:00 ... nauseous when you do that.

Sarah Guo 30:01 (laughs)

Dmitri Dolgov 30:01 Like, you kind of want to... You know, there's benefits on, uh, facing forward. But, you know, all of that, I think, will be for us as an industry to figure out as, you know, as we move forward. But I think the- the key- the key point, it becomes, you know, much more, like, the design is around the rider-

Sarah Guo 30:13 Mm-hmm.

Dmitri Dolgov 30:13 ... not around the driver.

Sarah Guo 30:14 It seems like it could do very interesting SKUs too where, you know, I've always wanted a car with like a Peloton in the back or something.

Dmitri Dolgov 30:19 Uh-huh.

Sarah Guo 30:19 So as you're commuting, you just kind of-

Dmitri Dolgov 30:22 ... connects with- I like that. I like it. Yeah. I wanna be in a bike, but I want my bike to be inside of a car. (laughs) I, I, I, there might be, like, let's see if we can-

Sarah Guo 30:28 The bike has to go 60 miles an hour.

Dmitri Dolgov 30:28 Yeah, why do you wanna be outside? Yeah, yeah, yeah. You know, it's a, a contained environment. Yeah. (laughs) Es- es- es- especially in California. (laughs) God, god forbid you actually, yeah, take a bike through my window. (laughs) Goodness. That's pretty good.

Sarah Guo 30:40 I wanna, I want a much less exotic form factor, man.

Dmitri Dolgov 30:42 (laughs)

Sarah Guo 30:42 I just wanna be able to take a Zoom with stable internet-

Dmitri Dolgov 30:44 Yeah.

Sarah Guo 30:44 ... and have it not look weird.

Dmitri Dolgov 30:45 On a bike, right? Where you wanna be on your bike.

Sarah Guo 30:47 On a bike, sure.

Dmitri Dolgov 30:48 Yeah, exactly.

Sarah Guo 30:48 On a bike a lot.

Dmitri Dolgov 30:49 And it's very exciting.

Sarah Guo 30:49 Yeah.

Dmitri Dolgov 30:50 Uh, we do- (laughs) We do increasingly have, you know, uh, team members

Dmitri Dolgov 30:54 calling into meetings from Waymos. Oh, yeah. So it is... (laughs) But it, it is, you know, actually, you know, like, uh, we, we joke about it, but it, uh, it gets to the point of privacy. Uh, it just becomes, you know, in this, if you don't have another human in the car, right? You can, you know, you can do a work meeting. You can do a, you know, call. You can, like, listen to your favorite music, uh, on full volume and not worry about, you know, that, that interaction of, like, having another human that you're sharing the spaces. So we are seeing, that was one of the, you know, hypotheses of the benefits, uh, of, of our product. And we are, you know, seeing very positive feedback from our Mm-hmm. ... RAs today, uh, uh, you know, along, along that specific dimension as well.

Sarah Guo 31:28 You know, as just somebody excited to see this technology expand coverage range, is the blocking factor to, let's say, you know, a billion miles a week, is it, like, putting more cars on the road from a capital perspective? Is it just operationally this can only happen so fast? Is it your view of, um, like, what you wanna see from a safety and trust perspective, like consumer trust perspective? What's the bottleneck?

Dmitri Dolgov 31:53 Primarily, it's the latter.

Sarah Guo 31:54 Okay.

Dmitri Dolgov 31:55 So we, we always, you know, our playbook has been to, uh, you know, go about it responsibly and gradually and-

Sarah Guo 32:01 Mm-hmm.

Dmitri Dolgov 32:01 ... earn trust every step of the way and have this transparent dialogue. Again, this is a very new thing, new technology, new product, you know, very different from what people are used to. I think it has to be this

Elad Gil 32:11 process. And again, you know, trust is this thing that's hard to, you know, earn-

Sarah Guo 32:16 Mm-hmm.

Dmitri Dolgov 32:16 ... but very easy to lose. So that's the main thing, right?

Sarah Guo 32:19 Mm-hmm.

Dmitri Dolgov 32:19 Uh, and you s- we see that. We see that, like, in places where, you know, we operate, uh, and we've engaged with the communities, and there's riders who have, you know, used Waymos. It is, uh, there's a lot of trust. There is, you know, people, you know, use the word magic a, a lot about the experience. Uh, and then you go to a different place where people have not experienced it, and there's more anxiety- Mm-hmm. ... and less trust. Mm-hmm. So you can't, uh, just get there in one step. You have to do it- Mm-hmm. ... kind of responsibly and iteratively. So that, that's, that's the main thing that

Elad Gil 32:47 Yeah, it sends us back when, uh, they had elevator operators, getting rid of the operator was a big deal, right? 'Cause you used to have somebody-

Dmitri Dolgov 32:52 Yeah.

Elad Gil 32:52 ... in the elevator who would close the door for you and push the button and control that experience. That's interesting to see that evolution of different types of technology over time or people's interpretation of it. Um, how do you think about generalized abilities? You mentioned you're building a general-purpose driver that could potentially port into other types of vehicles. Do you think there's other extensions into other forms of robotics with what you're building? Or do you think those are all more specialized models? Or how do you think about where this could go from that perspective?

Dmitri Dolgov 33:18 On the driving part, uh, we have kind of designed it to be generalizable, and we're v- very happy, you know, with what we're seeing, especially with the fifth-generation driver, and, like, the AI generalizes really well, you know, based on, again, we've been using, you know, data from a very broad ODD to build it, even, you know, if we're deploying- Mm-hmm. ... you know, responsibly and gradually once we, you know, believe that we've achieved the level of performance that we require for a certain ODD, which maps to, you know, certain

Dmitri Dolgov 33:42 areas and ZIP codes. And to get you the other part of the question of, you know, going beyond autonomous vehicles, some of the stuff, like the nature o- of the problem and the complexity, I think the, some of the research that we do, uh, is pretty foundational- Mm-hmm. ... uh, when we talk about, you know, perception, right? You know, you can be in a car. You can be in a different, uh, modality in a, like, operating in the physical world. Uh, a lot of the research that we've published and a lot of work we've done, I think can, you know, benefit those communities as well when we talk about, you know, AI, uh, being deployed in a kind of real-time system, in a safety-critical system. A lot of the work that we have done, I think, you know, can translate, uh, to others and, you know, so forth. When we talk about the evaluation of the system, kind of what, you know, many robotics applications, you know, beyond autonomous vehicles need a good realistic scalable simulator, you know, that, that, that, you know, fundamental work translates, so on, you know, so forth and so on. Mm-hmm. We are very focused on the- Mm-hmm. ... you know, trillions of miles where we can have the positive benefits. So for us, uh, like, I focus, I, I think focus is very, very important. So we are being very laser, you know, laser focused on, uh, driving.

Sarah Guo 34:43 Can I, um, ask, go back and ask, like, perhaps a, um, a more technical question here? Like, a while back, you said, um, you wanted to, you know, focus on the full autonomy problem. There were, uh, there are many other teams who actually, you know, have some lineage in the, like, you know, Waymo, Chauffeur, Google, um, programs that chose, uh, a use case that looked like it was going to be easier, trucking, like long-haul trucking, uh, deliveries. It's not clear that's much easier. Do you think there's a lesson to be learned here? Or, or at least,

Sarah Guo 35:18 you, you know, there are more miles being driven autonomously on the road in passenger vehicles by Waymo than in these other applications today.

Dmitri Dolgov 35:25 Yeah, uh-

Sarah Guo 35:26 What, what lesson is there to be learned?

Dmitri Dolgov 35:26 Yeah, no, no. Uh, I think that's, you know, great question. You know, kind of the big differentiation that I would draw, and this is, like, orders of magnitude, and the difference between full autonomy-

Sarah Guo 35:34 Mm-hmm.

Dmitri Dolgov 35:35 ... verse- at scale-

Sarah Guo 35:36 Mm-hmm.

Dmitri Dolgov 35:36 ... versus, uh, you know, a driver assist system.

Sarah Guo 35:39 Mm-hmm.

Dmitri Dolgov 35:40 Uh, and th- that, that's the big, like, that's the chasm. To your question, uh, of, you know, different vehicle platforms, different operating domain. You can have, you know, uh, slower speed, uh, applications where, you know, you do local deliveries, or you can have, you know, a trucking application, uh, on, you know, uh, freeways. Um, and they're a little bit different, but if we're talking about full autonomy, maybe there's second-order differences, but the first-order complexity is still there. Mm-hmm. Right? You can, uh, like, the, if you think about, you know, the core, the heart of the problem of, you know, building a generalizable and safe driver and, you know, being able to evaluate it, and the incredibly high bar of safety, the, you know, complexity of the noisy, messy physical environment and the long tail of, you know, people, you know, doing all kinds of, you know, weird thing, uh, and, uh, the necessity of making real-time decisions where milliseconds, you know, matter, and, like, how hard that AI problem is, uh, it's, you know, the, the, the distribution, the contours change a little bit if you're, you know, talking about, you know, freeways or lower speed. But the fundamentals are, like, you, you don't get, like, there's no silver bullets. You don't get to skip the, you know, core complexity. Mm-hmm. Now, for example, freeways, on a nominal case, they're a bit more structured.

Sarah Guo 36:44 Mm-hmm.

Dmitri Dolgov 36:44 Right? Like, you know, but, uh, you still, uh, encounter with lower frequency, but at higher speeds where the severity is high, you encounter-... all kinds of things. You encounter, you know, construction zones. You encounter, you know, grilles and mattresses and all kinds of stuff falling off of the cars in front of you. You encounter cars, uh, you know, having getting into accidents and kind of spinning out in front of you. You encounter, uh, you know, people driving recklessly, you know, uh, whether they're in cars, whether they're on motorcycles. You encounter, you know, pedestrians

Dmitri Dolgov 37:10 jaywalking. You encounter, you know, uh, all kinds of things, right? And it happens much less frequently. So this is where, you know, it might be unintuitive. If it happens, you know, at once per million miles, none of us have seen, you know, examples like that in our world, so it kind of, it can lead to this, you know, early stage optimism about like, okay, there's, there's the simplification. But if you wanna do it full autonomously, and you wanna do it at scale, that complexity is still there. It's just, you know, (inaudible) .

Sarah Guo 37:34 And why is it breaking from, like, um, you know, uh, let's say, advanced driver assistance that, uh, seems to work in more and more scenarios-

Dmitri Dolgov 37:42 Mm-hmm.

Sarah Guo 37:42 ... versus, let's say, full autonomy?

Dmitri Dolgov 37:45 What's the, what's the delta?

Sarah Guo 37:46 Yeah.

Dmitri Dolgov 37:47 It's the number of nines, right? And it, it's the nature of this, this problem, right? If you think about, you know, where we started in 2009.

Sarah Guo 37:53 Mm-hmm.

Dmitri Dolgov 37:54 Uh, one of our first, uh, you know, uh, milestones, one of the goals that we set for ourselves was to drive, you know, uh, 10 routes. Each one was 100 miles long, all over the Bay Area. Um, you know, freeways, uh, Downtown San Francisco, around Lake Tahoe, you know, everything. And you had to do 100 miles with no intervention. So the car had to, you know, drive autonomously from beginning to end. That's the goal that we created for ourselves. It was, you know, about a dozen of us. Took us maybe 18 months. We achieved that.

Sarah Guo 38:22 (laughs)

Dmitri Dolgov 38:22 2009.

Sarah Guo 38:23 (laughs)

Dmitri Dolgov 38:23 No ImageNet, no Continets, no transformers, no big models, com- tiny computers, you know, all this... Right? Very easy to get started. It's always been the property. And with every wave of technology, it's been al- you know, very easy, uh, to get started. But that, the hard problem, and it's kind of like, that, that early part of the curve has been getting, like, you know, even steeper and steeper.

Sarah Guo 38:41 Mm-hmm.

Dmitri Dolgov 38:42 But that's not where the complexity is. The complexity is in the long tail of the many, many, many nines. And you don't see that if you go, you know, uh, for a prototype, if you go for, you know, a driver assist system, uh, and this is where, you know, we've been spending all of our... That's the only hard part of the problem, right? And I guess, you know, nowadays, it's always been getting easier with every technical, uh, uh, kind of, uh, cycle. So nowadays, you can take with, like, all of the advances of... in AI, and especially in the, you know, generative AI world and the LLMs and VLMs, you can take kind of an almost off the shelf... Uh, you know, Transformers are amazing.

Sarah Guo 39:13 Mm-hmm.

Dmitri Dolgov 39:14 VLMs are amazing. You can take, uh, kind of a VLM, uh, that, uh, can accept, uh, images or video, and is, you know, has a decoder where you can give it, you know, text prompts and it will output text, and you can fine-tune it, you know, with just a little bit of, you know, data to go from, let's say, camera data in a car, to instead of words, to trajectories. Or, you know, whatever decisions you wanna make. You just, you know, take the thing 'cause a black box-

Sarah Guo 39:38 Mm-hmm.

Dmitri Dolgov 39:38 ... you know, you take whatever's been trained for

Elad Gil 39:40 You find too it a little bit and like not without, you know, 'cause I think if you ask any good grist in, in computer science to build, you know, NAV today, this is what they would do.

Sarah Guo 39:47 Yeah.

Dmitri Dolgov 39:47 And out of the box-

Sarah Guo 39:48 That's amazing, yeah.

Dmitri Dolgov 39:49 ... you get something that... It- it's amazing, right?

Sarah Guo 39:51 Yeah.

Dmitri Dolgov 39:51 Like, the power of Transformers-

Sarah Guo 39:52 Yeah.

Dmitri Dolgov 39:52 ... the power of VLMs is mind-blowing.

Sarah Guo 39:54 Mm-hmm.

Dmitri Dolgov 39:54 Right? So with just a little bit of effort, you get something on road, and it works. You can, you know, drive it on tens, hundreds of miles and it just, we'll, it will blow your mind. But then is that enough? Is that enough to remove the driver and drive, you know, millions of miles and have a safety record, you know, that is demonstrably better than humans? No. Right? And I guess this is, you know, with every tech, you know, evolution of technology and any breakthrough in AI, we've seen kind of that reshaping.

Sarah Guo 40:16 Is it the right way to think of the, um, the iteration cycle for Waymo now still like many other AI companies where an eval, some, some set of cases comes up that you don't handle as well as you want, and then you collect more data, and you put it into the pipeline, you retrain, and you deploy? Or are there still architectural changes that are happening even past this point of cracking the nut?

Dmitri Dolgov 40:37 Both. Uh, so the first thing you mentioned where it's, you know, the, the data collection and, uh, you know, understanding where performance is not good enough, kind of building the whole, you know, data flywheel and evaluation flywheel? That's at the heart of it.

Sarah Guo 40:48 Mm-hmm.

Dmitri Dolgov 40:48 Right? But, uh, I think there is, this is, and this is where it gets, you know, a bit nuanced. You know, what do you do? Wha- what is the architecture, and what is the training methodology, right? Uh, in particular, uh, kind of the simplest thing you can do is, you know, an end-to-end model that is trained just on imitative, you know, kind of imitating human drivers.

Sarah Guo 41:06 Mm-hmm.

Dmitri Dolgov 41:06 So, you know, very easy. Sensors, you know, pixels go in, driving, you know, behavior that you have examples of, and you just train it to imitate human drivers. And you can run this, you know, flywheel and kind of run the circle that you described while operating, you know, in a, under, you know, that paradigm.

Sarah Guo 41:21 Yeah.

Dmitri Dolgov 41:21 And you'll make progress, right? It's a very well-understood kind of approach, right? Uh, you balance your data. You find some examples, you know, where you're not performing as well as you would've liked, if you figured out, you know, how to evaluate it. Uh, and then-

Sarah Guo 41:32 You stimulate more examples that look like that.

Dmitri Dolgov 41:33 Well, okay, now you're getting simulation, right?

Sarah Guo 41:35 Okay. (laughs)

Dmitri Dolgov 41:35 But this is, this is, this is where we get, like, exactly how it starts to get interesting, right?

Sarah Guo 41:38 Yeah.

Dmitri Dolgov 41:38 'Cause, uh, you know, for what you described, you don't need a simulator. You can run that, you know, you can turn the crank on that, this whole machinery even without having a simulator, just do the open loop to imitate. So you find more examples where, you know, the, you, like, of example things you wanna imitate. And you reduce the number. You find examples where, you know, humans did something you don't like, so you, you know, you kind of, you, you, this is, you know, your dataset balancing. And you will, you know, continue to improve. You might plateau in the wrong place. You might plateau in the right place for a driver assist system. You will plateau at the wrong place for a fully autonomous system. So then you need to kind of, you know, build that machinery. You'll still, like, the, at a high level, that principle of new data and, you know, augmentation, uh, holds. But to, you know, really go the distance to full autonomy, you need to do other things, right? You need to do synthetic data. You need to do closed-loop simulation. Maybe sensor simulation is not enough because when you do it at scale, it's just highly, you know, inefficient. It's not practical, right, just kind of doing, you know, imitating stuffs and piping it through the whole thing. So then you get into, like, an immediate representations. Can you, you know, uh, simulate in that space? And you still are doing kind of at a very high level, you know, the, the flywheel-

Sarah Guo 42:35 The loop, yeah.

Dmitri Dolgov 42:35 ... the loop, the loop holes, right?

Sarah Guo 42:36 Yeah.

Dmitri Dolgov 42:36 But what's in the loop, I think it depends on-

Sarah Guo 42:39 You have to get more creative.

Dmitri Dolgov 42:39 Exactly, exactly.

Sarah Guo 42:40 Mm-hmm.

Elad Gil 42:40 So I guess where are we today, and where do you think we're going?

Dmitri Dolgov 42:44 I think we are, you know, reflecting on kind of the, this journey, uh, that's been quite a few years. I find myself, I think, more excited than ever about

Dmitri Dolgov 42:58 where we are, the momentum, and the future. Right? And I've been doing this, well, for close to two decades. Uh, the vision was always there, all right? But we had these big existential questions. Can we build the thing? Can we, you know, figure out what's good enough, and how do we evaluate it? Uh, can... will people wanna use it? Can we do it in a way that's, you know, commercially viable, you know, uh, and so forth and so on? Uh, and, you know, can we go the distance? And like now, where we are today, you know, operating at the scale we are and scaling ?1:50:03 ] we've, we've, we've demonstrated that we can build a thing. We are proud of our safety record, figured out how to evaluate it. We see that people wanna use it, and we get very positive feedback, and people are excited about it. We see that we can do it in a way that's cost-efficient and kind of likely, you know, viable. Uh, so I am super excited about, you know, what the future holds, and we're starting to talk about realizing the mission of actually, you know, making, uh, you know, realizing those safety benefits. So now it's all about, you know, optimization, scaling, and bringing this technology to more people and more places.

Elad Gil 44:01 Amazing. Yeah, very exciting. Thank you again for joining us today.

Dmitri Dolgov 44:04 Thank you for having me.

Sarah Guo 44:04 And congratulations-

Elad Gil 44:05 Thanks for the, really good-

Sarah Guo 44:06 ... on, uh, you know, the breakthrough progress over the last, you know, small amount of time.

Dmitri Dolgov 44:09 Thank you. Thank you. (instrumental music)

Sarah Guo 44:12 Find us on Twitter @nopriorspod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.