David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI | Lex Fridman Podcast #44

Lex Fridman 0:00 The following is a conversation with David Ferrucci. He led the team that built Watson, the IBM question-and-answering system that beat the top humans in the world at the game of Jeopardy. From spending a couple of hours with David, I saw a genuine passion, not only for abstract understanding of intelligence, but for engineering it to solve real-world problems under real-world deadlines and resource constraints. Where science meets engineering is where brilliant, simple ingenuity emerges. People who work at joining the two have a lot of wisdom earned through failures and eventual success. David is also the founder, CEO, and chief scientist of Elemental Cognition, a company working to engineer AI systems that understand the world the way people do. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D-M-A-N. And now, here's my conversation with David Ferrucci.

Lex Fridman 1:06 Your undergrad was in biology with a- with an eye toward medical school before you went on for the PhD in computer science. So let me ask you an easy question. What is the difference between biological systems and computer systems? In your... when you sit back, look at the stars, and think philosophically.

David Ferrucci 1:28 I often wonder, I often wonder whether or not there is a- a substantive difference. I mean, I think the thing that got me into computer science and into artificial intelligence was exactly this presupposition that, uh, if we can get machines to think, or I should say this question, this philosophical question, if we can get machines to think, to understand, to process information the way do- we do, so if we can describe a procedure or describe a process, even if that process were the intelligence process itself, then what would be the difference? So, um, from a philosophical standpoint, I'm not sure I'm convinced that there- there- there is. I mean, you can go in the direction of spirituality or you can go in the direction of a soul, but in terms of, you know, what we can- what we can experience, uh, from an intellectual and physical perspective, I'm not sure there is. Clearly, there implement- there- there are different implementations. But if you were to say, as a biological information, processing system fundamentally more capable than one we might be able to build out of silicon or- or some other, uh, substrate, uh, I don't- I don't know that there is.

Lex Fridman 2:46 How distant do you think is the biological implementation? So fundamentally, they may have the same capabilities, but is it, um, really a far mystery where a huge number of breakthroughs are needed to be able to understand it? Or is it something that, for the most part, in the important aspects, echoes of the same kind of characteristics?

David Ferrucci 3:11 Yeah, that's interesting. I mean, uh, so, you know, your question presupposes that there's this goal to recreate, you know, what we perceive as biological intelligence. I'm not- I'm not sure that's the- I'm not sure that- that's how I would state the goal. I mean, I think that studying-

Lex Fridman 3:27 What is the goal?

David Ferrucci 3:29 Good. So I think there are a few goals. I think that understanding the human brain and how it works is important for us to be able to diagnose and treat issues, for us to understand our own strengths and weaknesses, um, both intellectual, psychological, and physical. So neuroscience and understanding the brain from that perspective has a ne- there's a clear, clear goal there. From the perspective of saying I want to m- I want to- I want to mimic human intelligence, that one's a little bit more interesting. Human intelligence certainly has, um, a lot of things we envy. It's also got a lot of problems too. So I think we're capable of sort of stepping back and saying, "What do we want out of it? Uh, what do we want out of an intelligence? Uh, how do we want to communicate with that intelligence? How do we want it to behave? How do we want it to perform?" Now, of course, it's- it's- it's somewhat of an interesting argument because I'm sitting here as a human with a biological brain and I'm critiquing the strengths and weaknesses of human intelligence and saying that we have the capacity to s- the capacity to step back and say, "Gee, what do- what is intelligence and what do we really want out of it?" And that even- in and of itself suggests that human intelligence is something quite enviable, that it could- it- you know, it can- it can- it can, um, introspect that- it can introspect that way.

Lex Fridman 4:58 And the flaws, you mentioned the flaws. That humans have flaws.

David Ferrucci 5:00 Yeah. But I think- I think that flaws that human intelligence has is ex- extremely, um, prejudicial and biased in the way it draws many inferences.

Lex Fridman 5:10 Do you think those are... Sorry to interrupt. Do you think those are features or are those bugs? Do you think the- the prejudice, the forgetfulness, the fear... What other flaws? List them all. What? Love? Maybe that's a flaw. Do you think those are all things that can be get- gotten- get in the way of intelligence or the essential components of intelligence?

David Ferrucci 5:33 Well, again, it's- i- if you go back and you define intelligence as being able to sort of accuracy- accurately, precisely, rigorously reason, develop answers, and justify those answers in an objective way, yeah, then human intelligence has these flaws in that it tends to be more influenced by some of the things you said.

Lex Fridman 5:55 Mm-hmm.

David Ferrucci 5:56 Uh, and it's- and it's largely an inductive process, meaning it takes past data, uses that to predict the future.... very advantageous in some cases, but fundamentally biased and prejudicial in other cases, 'cause it's gonna be strongly influenced by its priors, whether they're f- whether they're right or wrong for some, you know, objective reasoning perspective. You're gonna favor them because that's, those are the decisions or those are the paths that succeeded in the past. And I think that mode of intelligence makes a lot of sense for, um, when your primary goal is to act quickly and s- and, and survive and make fast decisions. And I think those create problems, uh, when you wanna think more deeply and make more objective and reasoned decisions. Of course, humans capable of doing both.

Lex Fridman 6:48 Right.

David Ferrucci 6:48 They do sort of one more naturally than they do the other, but they're capable of doing both.

Lex Fridman 6:53 You's saying they do the one that responds quickly and it more naturally?

David Ferrucci 6:56 Right.

Lex Fridman 6:57 'Cause that's the thing we kinda need to not be eaten by, uh, the p- the predators-

David Ferrucci 7:02 Well-

Lex Fridman 7:02 ... in the world.

David Ferrucci 7:02 ... for example, but I mean, but, uh, then we, we've, we've learned to reason, uh, through logic. We've developed science. We've trained people to do that. I think that's harder for the individual to do. Uh, I think it requires training and, you know, and, and, and teaching. I think we are... human mind is cer- certainly is capable of it, but we find it more difficult. And then there are other weaknesses, if you will, as you mentioned earlier, just memory capacity and, and, um, how many chains of inference can you actually, um, go through without, like, losing your way, so just focus and...

Lex Fridman 7:40 S- so the way you think about intelligence, and we're really sort of floating in this philosophical s- slightly space, but I think you're, like, the perfect person to talk about this because, uh, we'll get to Jeopardy! and beyond, th- that's like an incredible, one of the most incredible accomplishments in AI, in the history of AI, but hence, the philosophical discussion.

Lex Fridman 8:03 So let me ask, you've kind of alluded to it, but let me ask again, what is intelligence underlying the discussions we'll have with, with Jeopardy! and beyond, how do you think about intelligence? Is it a sufficiently complicated problem, being able to reason your way through solving that problem? Is that kinda how you think about what it means to be intelligent?

David Ferrucci 8:25 So I, I think of intelligence two, primarily two ways. One is the ability to predict. So in other words, if I have a problem, what's gonna... can I predict what's gonna happen next? Whether it's to, you know, predict the answer of a question or to say, "Look, I'm looking at all the market dynamics and I'm gonna tell you what's gonna happen next," or you're in a, in a room and somebody walks in and you're gonna predict what they're gonna do next or what they're gonna say next.

Lex Fridman 8:53 So in a, in a highly dynamic environment full of uncertainty, be able to-

David Ferrucci 8:57 Lots of, lot-

Lex Fridman 8:57 ... predict.

David Ferrucci 8:58 ... you know, the more-

Lex Fridman 8:59 Yeah.

David Ferrucci 8:59 ... the more variables, the more complex, the more possibilities, the more complex. But can I take a small amount of prior data and learn the pattern and then predict what's gonna happen next accurately and consistently? That's a f- that's certainly a form of intelligence.

Lex Fridman 9:16 W- what do you need for that, by the way? You need to have an understanding of the way the world works in order to be able to unroll it into the future, right? Like, w- what do you think-

David Ferrucci 9:26 Well-

Lex Fridman 9:26 ... is needed to predict?

David Ferrucci 9:27 ... depends what you mean by understanding. I, I, I, I, I need to be able to find that function. This is very much like what-

Lex Fridman 9:33 It's a function.

David Ferrucci 9:34 ... deep learning does, machine learning does, is if you give me enough prior data and you tell me what the output variable is that matters, I'm gonna sit there and be able to predict it. And if I can predicu- predict it accurately so that I can get it right more often than not, I'm smart. If I can do that with less data (laughs) and less training time, I'm even smarter. If I can figure out what's even worth predicting, (laughs) um, I'm smarter, meaning I'm at... I'm, I'm figuring out what path is gonna get me toward a goal.

Lex Fridman 10:06 What about picking a goal? Sorry to interrupt again.

David Ferrucci 10:08 That's... Well, that's the interesting about picking a goal, sort of an interesting thing, and I think that's where you bring in what are you pre-programmed to do? We talk about humans and, well, humans are pre-programmed to survive. So sort of their primary, you know, driving goal, what do they have to do to do that? And that, that could be very complex, right? So it's not just, it's not just figuring out that you need to run away from the ferocious tiger, but we survive in s- a social context, as an example. So understanding the subtleties of social dynamics becomes something that's important for surviving, finding a mate, reproducing, right? So we're continually challenged with complex sets of variables, complex constraints, rules if you will, that we, we... or patterns, and we learn how to find the functions and predict the things, in other words, represent those patterns efficiently, and be able to predict what's gonna happen, and that's a form of intelligence. That doesn't really recoi- that doesn't really require anything specific other than the (laughs) ability to find that function and, and predict that right answer. It's certainly a form of intelligence. But then when we, we, when we say, "Well, do we understand each other?" In other words, um, do... would you perceive me as, as intelligent beyond that ability to predict? So now I can predict, but I can't really articulate h- how I'm going through that process, what my underlying theory is for predicting, and I can't get you to understand what I'm doing so that you can follow... you can figure out how to do this yourself if you hadn't r- if you did not have, for example, the right pattern-matching machinery that I did. And now we ha- potentially have this breakdown where, in effect, I'm intelligent, but I'm sort of an alien intelligence relative to, to you.

Lex Fridman 12:02 (laughs) You're intelligent, but nobody knows about it.... uh, or the-

David Ferrucci 12:05 Well, I can see-

Lex Fridman 12:06 ... or I can't-

David Ferrucci 12:06 ... I can see the, I can see the output, like-

Lex Fridman 12:08 So, so you're saying, let's sort of separate the two things. One is you explaining why you were able to predict the future and, and, uh, and the second is me being able to, like, impressing me that you're intelligent, me being able to know that you successfully predicted the future. Do you think that's...

David Ferrucci 12:29 Well, it's not impressing you that I'm intelligent. In other words, you may be convinced that I'm intelligent in some form.

Lex Fridman 12:36 So how, what would convince-

David Ferrucci 12:37 Because of my ability to predict.

Lex Fridman 12:38 So I would look at the metrics.

David Ferrucci 12:40 When you can, I just say, "Wow."

Lex Fridman 12:41 "Wow."

David Ferrucci 12:41 "You're right all, you're, you're, you're right more times than I am. You're doing something interesting." That's a form of, that's a form of intelligence. But then what happens is, if I say, "How are you doing that?" and you can't communicate with me, and you can't describe that to me, now I may lab- label you a savant. I may, I may say, "Well, you're doing something weird and it's, and it's just not very interesting to me, because you and I can't really communicate." And, and so now the, the, so this is interesting, right? Because now this is, you're in this weird place where for you to be recognized as intelligent the way I'm intelligent-

Lex Fridman 13:20 Right.

David Ferrucci 13:21 ... then you and I sort of have to be able to communicate. And then my, we start to understand each other, and then my respect and my, my appreciation, my ability to relate to you starts to change. So now you're not an alien intelligence anymore. You're, you're a human intelligence now, because you c- you and I can communicate. And so I think when we look at, when we look at a- when we look at animals, for example, animals can do things we can't quite comprehend. We don't quite know how they do them, but they can't really communicate with us. They can't put what they're going through in our terms, and so we think of them as sort of, "Well, they're these alien intelligences and they're not really worth necessarily what we're worth." We don't treat them the same way as a result of that. But it's, it's hard because who knows what, what, you know, what's going on.

Lex Fridman 14:11 So just a, a quick elaboration on that. The, explaining that you're intelligent, the, explaining the, the reasoning that went into the prediction is not some kind of mathematical proof. Uh, if we look at humans, look at political debates and discourse on Twitter, uh, it's mostly just telling stories. So you, your-

David Ferrucci 14:36 That's a f-

Lex Fridman 14:36 ... your task is, sorry. That y- your task is not to tell an accurate depiction of how you reason, but to tell a story, real or not, that convinces me that there was a mechanism by which you arrived-

David Ferrucci 14:52 Ultimately, that's what a proof is. I mean, even a mathematical proof is, is that. Because ultimately, the other mathematicians have to be convinced (laughs) by your proof.

Lex Fridman 14:59 (laughs)

David Ferrucci 15:00 Um, otherwi- otherwise... In fact, there have been-

Lex Fridman 15:03 That's the metric of success, yeah.

David Ferrucci 15:04 ... several proofs. There have been, yeah. There have been several proofs out there where mathematicians would study it for a long time before they were convinced that it actually proved anything.

Lex Fridman 15:10 (laughs)

David Ferrucci 15:10 Right? You never know if it proved anything until the community of mathematicians decided that it did. So I mean, so, yeah, it's, but it's, it's a real thing, right? And, and, and that's sort of the point, right? Is that ultimately un- you know, this notion of understanding, us understanding s- something is ultimately a social concept.

Lex Fridman 15:28 Yes.

David Ferrucci 15:28 In other words, you, you, I have to convince enough people that I, I did this in a reasonable way. I could do this in a way that other people can understand and, and replicate, and, um, that makes sense to them. So we're ver- we're, our human intelligence is bound together in, in that way. We're bound up, um, in that sense. We sort of never really get away with it until we can con- sort of convince others, uh, that our thinking process, you know, makes sense.

Lex Fridman 15:55 Do you think the general question of intelligence is then also a social construct? So if we task, ask questions of an artificial intelligence system, "Is this system intelligent?" the answer will ultimately be a socially constructed concept.

David Ferrucci 16:12 I think, I think, so I, so I think, yeah, I'm making two statements. I'm saying we can try to define intelligence in a super objective way that says, "Here, here's this data. I wanna predict this type of thing."

Lex Fridman 16:25 Right.

David Ferrucci 16:25 "Um, learn this function, and then if you get it right, often enough, we consider you intelligent."

Lex Fridman 16:32 But that's more like a savant.

David Ferrucci 16:34 More like a question of intelligence.

Lex Fridman 16:34 That's-

David Ferrucci 16:34 I think it i-

Lex Fridman 16:35 Yeah.

David Ferrucci 16:35 ... I think it is. It doesn't mean it's not useful. It's f-

Lex Fridman 16:37 Right.

David Ferrucci 16:37 ... it could be incredibly useful. It could be solving a problem we can't otherwise solve, and, um, can solve it more reliably than we can. But then there's this notion of can humans take responsibility for the, the, the decision that you're, that you're making? Can we make those decisions ourselves? Can we relate to the process that you're going through? And now you, as an agent, whether you're a machine or another human, frankly, are now obliged to make me understand how it is that you're arriving at that answer, and allow me, me, me or the, obviously, a community or a judge of people to decide whether or not, whether or not that makes sense. And by the way, that happens with humans as well.

Lex Fridman 17:19 Right.

David Ferrucci 17:20 You're sitting down with your staff, for example, and you ask for suggestions about what to do next, and someone says, "Oh, I think you should buy, and I s- think you should buy this much," or whatever, or sell or whatever it is. Or I s- think you should launch the product today or tomorrow, or launch this product versus that product, whatever the decision may be. And you ask, "Why?" And the person said, "I just have a good feeling about it."

Lex Fridman 17:42 Yeah.

David Ferrucci 17:42 And it's not, you're not very satisfied. Now, that person could be, you know, you might s- say, "Well, you've been right, you know, before, but I'm gonna put the company on the line. Can you explain to me why I should believe this?"

Lex Fridman 17:56 Right. And that explanation may have nothing to do with the truth. You just, the, the ultim-

David Ferrucci 18:01 It's gotta convince the other person.

Lex Fridman 18:03 The ultimate metric is-

David Ferrucci 18:04 It could still be wrong.

Lex Fridman 18:04 Yeah.

David Ferrucci 18:04 It could still be wrong.

Lex Fridman 18:05 It's just gotta be convincing. But it's got, it's ultimately gotta be convincing.

David Ferrucci 18:07 So-

Lex Fridman 18:07 And that's why I'm saying i- i- it's...

David Ferrucci 18:10 ... we're, we're bound together.

Lex Fridman 18:11 Yeah.

David Ferrucci 18:11 Right? Our intelligences are bound together in that sense. We have to understand each other. And, and if, for example, you're giving me an explanation, I mean, this is a very important point, right? You're giving me an explanation and I'm, and I'm, and I, and I have, I, I'm not good, and then I, I'm not good at reasoning well and being objective and following logical paths and consistent paths, and I'm not good at measuring and sort of computing probabilities across those paths, what happens is collectively, we're not gonna do, we're not gonna do well.

Lex Fridman 18:50 How hard is that problem, the second one? So, we'll, I think we'll talk quite a bit about the, the, the first on a specific objective metric benchmark performing well, but being able to explain the steps of the reasoning, how hard is that problem?

David Ferrucci 19:10 I think that's, I think that's very hard. I mean, I think that that's, um... Well, it's hard for humans.

Lex Fridman 19:18 The thing that's hard for humans, as you know, may not necessarily be hard for computers and vice versa.

David Ferrucci 19:23 That's right.

Lex Fridman 19:24 So, sorry, so (laughs) ho- how hard is that problem for computers?

David Ferrucci 19:31 I think it's hard for computers and the reason why I related to or say saying that it's also hard for humans is because I think when we step back and we say we wanna design computers to do that, one of the things we have to recognize is we're not sure how to do it well.

Lex Fridman 19:50 Right.

David Ferrucci 19:50 I'm not sure we have a recipe for that and e- e- even if you wanted to learn it, it's not clear exactly what data we use and what judgments we use to learn that well. And so what I mean by that is, if you look at the entire enterprise of science, science is supposed to be about objective reasonant- reason, right? So, when you think about gee, who's the most intelligent person or group of people in the world, do we think about the savants who can close their eyes and give you a number or do you think about the think tanks or the scientists or the philosophers who kinda work through, uh, the details and write the papers and come up with the thoughtful, logical proofs and use the scientific method? And I think it's the latter. And my point is that how do you train someone to do that? And that's what I mean by it's hard. How do you-

Lex Fridman 20:47 Hm.

David Ferrucci 20:47 What's the process of training people to do that well? That's a hard process. We work, as a society, we work pretty hard to get other people to understand our thinking and to convince them of things. Now we could persua- persuade them, obviously we talked about this, like, human flaws or weaknesses we can persuade through m- persuade them through emotional means.

Lex Fridman 21:11 Mm-hmm.

David Ferrucci 21:12 But to, but to get them to understand and connect to and follow a logical argument is difficult. We try it, we do it, we do it as scientists, we try to do it as journalists, we, you know, we try to do it as, you know, even artists in many forms, as writers, as teachers, we go through a fairly significant training process, uh, to do that. And then we could ask, "Well, why is that so hard?" But it's hard and for humans it takes a lot of work and when we step back and say, well, step back and say, "Well, how do we get a machine to, how do we get a machine to do that?" It's a vexing question.

Lex Fridman 21:52 How would you begin to try to solve that? And maybe just a quick pause, because there's an optimistic notion in the things you're describing, which is r- uh, being able to explain something through reason. But if you look at algorithms that recommend things that we'll look at next, whether it's Facebook, Google, advertisement-based companies, you know, their goal is to convince you to buy things based on anything. Uh, so that could be reason 'cause the best of advertisement is showing you things that you really do need and explain why you need it. But it could also be through emotional manipulation. The algorithm that describes why a certain reason, a certain decision was, was made, how hard is it to do it w- through emotional manipulation? And why is that a good or a bad thing? So you've kind of focused on reason, logic-

David Ferrucci 22:56 Mm-hmm.

Lex Fridman 22:57 ... really showing in a, in a clear way why something is good. One, is that even a thing that us humans do? (laughs) And, uh, and, and two, how do you think is the difference between, um, the reasoning aspect and the emotional manipulation?

David Ferrucci 23:14 Well, you know, so you call it emotional manipulation but more, more objectively is essentially saying, you know, thing, you know, there are certain features of things that seem to attract your attention. I mean, they kinda give you more of that stuff.

Lex Fridman 23:26 Manipulation is a bad word.

David Ferrucci 23:28 Yeah. I mean, I'm not-

Lex Fridman 23:29 There might be more.

David Ferrucci 23:29 ... saying it's good, right or wrong, I mean, is, uh, uh, it works to get your attention and it works to get you to buy stuff, and when you think about algorithms that look at the patterns of the, you know, patterns of features that you seem to be spending your money on and say, "I'm gonna give you something with a similar pattern," so I'm gonna learn that function because the objective is to get you to click on it or get you to buy it or whatever it is. I don't know, I mean, that, it is, like, it is what it is. I mean, that's what the algorithm does. You can argue whether it's good or bad. It depends what your, you know, what your, what your goal is.

Lex Fridman 24:00 I guess this seems to be-

David Ferrucci 24:01 It is g-

Lex Fridman 24:01 ... very useful for convincing or telling a story.

David Ferrucci 24:05 It, for convincing, for convincing humans-

Lex Fridman 24:06 Yeah.

David Ferrucci 24:07 ... it's good because you, because again, this goes back to how does a hu- you know, what, what is the human behavior like? How does, what does, how, w- how does the human, you know, brain respond to things? Um, I think there's a more optimistic view of that too, which is...... that if you're searching for a certain kinds of things, you've already reasoned that you need them and these, these algorithms are saying, "Look, that's up to you. The reason whether you need something or not, that's your job." You know, you, you may, you may have an unhealthy addiction to this stuff, or you may have a reasoned and thoughtful explanation for why it's important to you. And the algorithms are saying, "Hey, that's, like, whatever. Like, that's your problem. All I know is you're buying stuff like that, you're interested in stuff like that. Could be a bad reason, could be a good reason. That's up to you. I'm gonna show you more of that stuff." And so... And I, and I, and I think that that's... It's not good or bad. It, it's not reasoned or not reasoned. The algorithm is doing what it does, which is saying, "You seem to be interested in this. I'm gonna show you more of that stuff." And I think we're seeing this not just in buying stuff, but even in social media. You're reading this kind of stuff. I'm not judging on whether it's good or bad. I'm not reasoning at all. I'm just saying, "I'm gonna show you other stuff with similar features." And, you know, and, like, and that's it. And I wash my hands from it and I say, "That's all," you know, "that's all that's going on."

Lex Fridman 25:26 You know, there is... Uh, you know, people are so harsh on AI systems. So one, the bar of performance is extremely high, and yet we also ask them to, uh, in the case of social media, to, uh, help find the better angels of our nature and help make a better society. What do you think about the role of AI there?

David Ferrucci 25:47 So that's, so that's... I agree with you. That's, that's the interesting dichotomy, right? Because on one hand, we're sitting there and we're sort of doing the easy part, which is finding the patterns. We're not building a... The system's not building a, a theory-

Lex Fridman 26:01 Right.

David Ferrucci 26:01 ... that is consumable and understandable by other humans that can be explained and justified. And, and so on one hand to say, "Oh, you know, AI is doing this. Why isn't it doing this other thing?" Well, this other thing is a lot harder. And it's interesting to think about why, why, why it's harder. And bec- because you're interpreting, you're interpreting the data in the context of prior models. In other words, understandings of what's important in the world, what's not important. What are all the other abstract features that drive, um, our decision-making? What's sensible? What's not sensible? What's good? What's bad? What's moral? What's valuable? What isn't? Where is that stuff? No one's applying the interpretation. So when I, when I see you clicking on a bunch of stuff and I look at these simple features, the raw features, the features that are there in the data, like what words are being used or how long the material is, um, or other very superficial features. What colors are being used in the material? Like, I don't know why you're clicking on the stuff you're clicking. Or if it's products, what the pri- what the price is or what... the categories and stuff like that. And I just feed you more of the same stuff. That's very different than kind of getting in there and saying, "What does this mean?" What, what... The stuff you're reading, like, why are you reading it? What assumptions are you bringing to the table? Are those assumptions sensible? Uh, is the mat- does the material make any sense? Does it, does it lead you to thoughtful, good conclusions? Again, there's judgment, there's interpretation and judgment involved in that process that isn't really happening in, in, in the AI today. That's harder-

Lex Fridman 27:45 Right.

David Ferrucci 27:45 ... because you have to start getting at the meaning of this... of the, of the stuff, of the content. You have to get at how humans interpret the content-

Lex Fridman 27:55 Right.

David Ferrucci 27:56 ... relative to their value system and deeper thought processes.

Lex Fridman 28:00 So that's what meaning means, is not just some kind of deep, timeless, semantic thing that the statement represents, but also how a large number of people are likely to interpret. So it's, again, even meaning is a social construct. So you have to try to predict how most people would understand this kind of statement.

David Ferrucci 28:24 Yeah. Meaning is often rel- relative, but, uh, m- meaning implies that the connections go beneath the surface of the artifacts. If I show you a painting, it's a bunch of colors on a canvas. What does it mean to you?

Lex Fridman 28:36 Yeah.

David Ferrucci 28:37 And it may mean different things to different people because of their different experiences. It may mean something even different to the artist who, who painted it. As we try to get more rigorous with our communication, we try to really nail down that meaning. So we go from abstract art to precise mathematics, precise engineering drawings, and things like that. We're really trying to say, "I want to narrow that, that space of possible interpretations," because the precision of the communication ends up becoming more and more important. And so that means that I have to specify, and I think that's why this becomes really hard. Because if I'm just showing you an artifact and you're looking at it superficially, whether it's a bunch of words on a page or whether it's, um, you know, brushstrokes on a canvas or pixels in a photograph, you can sit there and you can interpret lots of different ways at many, many different levels. Um, but when I wanna, when I want to, uh, align our understanding of that, I have to specify a lot more stuff that's actually not in, uh, not directly in the artifact. Now I have to say, "Well, how do you wer- how are you interpreting this image and that image? And what about the colors and what do they mean to you? What's, what perspective are you bringing to the table? What are your prior experiences with those artifacts? What are your fundamental assumptions and values? What, what is your ability to kind of reason, to chain together, um, logical implications as you're sitting there and saying, 'Well, if this is the case, then I would conclude this. And if that's the case, then I would conclude that'?" And, uh... So, your reasoning processes and how they work, your prior models and, uh, what they are, your values and your assumptions. All those things now come together into the interpretation. Getting in sync on that is, is hard.

Lex Fridman 30:34 And yet, humans are able to intuit some of that without any pre-

David Ferrucci 30:39 Because they have the shared experience.

Lex Fridman 30:41 ... mea- and we're not talking about shared, two people having a shared experience.

David Ferrucci 30:44 No.

Lex Fridman 30:44 We, as a society-

David Ferrucci 30:45 That's correct. We have the shared experience and we have similar brains, so we tend to inter- so in other words, part of our shared experience is our shared local experience. Like, we may live in the same culture, we may live in the same society, and therefore, we have similar educations. We have similar qu- what we like to call prior models, about the world, prior experiences, and we use that as a... Think of it as a wide collection of interrelated variables, and they're all bound to similar things. And so we take that as our background, and we start interpreting things similarly. But as humans, we have a, we have, um, a lot of shared experience. We do have similar brains, similar goals, similar emotions, under similar circumstances, because we're both humans. So now, one of the early questions you asked, well, how is biological and g- you know, computer, uh, information systems, uh, fundamentally different? Well, one i- you know, one is, can... Sh- humans come with a lot of pre-programmed stuff.

Lex Fridman 31:43 Yeah.

David Ferrucci 31:43 A ton of pro- programmed stuff. And they're able to communicate because they have a lot of... Because they

David Ferrucci 31:49 share that stuff.

Lex Fridman 31:50 Do you think that shared knowledge... If, if, if we can maybe escape the hardware question, how much is encoded in the hardware? Just the shared knowledge and the software, the, the history, the many centuries of wars and so on that, that came to today. That shared knowledge. Uh, how hard is it to encode? And did you have a hope? Can you speak to how hard is it to encode that knowledge systematically in a way that could be used by a computer?

David Ferrucci 32:22 So I think it is possible to learn to, for a machine, to program a machine to acquire that knowledge with a similar foundation. In other words, an inter- a similar interpretative, interpretative foundation for processing that knowledge.

Lex Fridman 32:38 Uh, what do you mean by that? How-

David Ferrucci 32:39 So in other, in other words-

Lex Fridman 32:40 ... foundation?

David Ferrucci 32:40 ... we view the world in a particular way. And so, in other words, we, we have a, if you will, as humans, we have a framework for interpreting the world around us.

Lex Fridman 32:51 Mm-hmm.

David Ferrucci 32:52 So we have multiple frameworks for interpreting the world around us. But, uh, if you're interpreting, for example, socio-political interactions, you're thinking about, well, there's people, there's collections and groups of people. They have goals. Goals largely built around survival and quality of life.

Lex Fridman 33:10 Mm-hmm.

David Ferrucci 33:10 There are e- there are fundamental economics around scarcity of resources. And when, when humans come and start interpreting a situation like that, because you brought, you brought up, like, historical events. They start interpreting situations like that. They apply a lot of this, a lot of this, this fundamental framework for interpreting that. Well, who are the people? What were their goals? What resources did they have? How much power or influence did they have over the other... Like, just fundamental-

Lex Fridman 33:38 Yeah.

David Ferrucci 33:38 ... substrate, if you will, for interpreting and reasoning about that. So I think it is possible to imbue a computer with that, that stuff that humans, like, take for granted when they go and, and, and sit down and try to interpret things. And then, and then with that, with that foundation, they acquire, they start acquiring the details, the specifics in any given situation, are then able to interpret it with regard to that framework. And then given that interpretation, they can do what? They can predict. But not only can they predict. They can predict now with an explanation that can be given in those terms, in the terms of that underlying framework that most humans share.

Lex Fridman 34:21 Mm-hmm.

David Ferrucci 34:22 Now, you could find humans that come and interpret events very differently than other humans, because they're, like, using a, a different s- different framework. You know, the movie Matrix comes to mind, where, you know, they decided that humans were really just batteries, and that's how they (laughs) interpreted the value of humans-

Lex Fridman 34:39 Mm-hmm.

David Ferrucci 34:39 ... um, as a source of electrical energy. So but, um, but I think that, you know, for the most part, we, we, we have a way of, of interpreting the events or at least social events around us, because we have this shared framework. It comes from, again, the fact that we're, we're similar beings that have similar goals, similar emotions, and we as... We can make sense out of these. These frameworks make sense to us.

Lex Fridman 35:05 So how much knowledge is there, do you think? So it's... You said it's possible.

David Ferrucci 35:09 Well, there's always a tremendous amount of detailed knowledge in the world. There are, you know. You can imagine, you know, effectively infinite number of unique situations and unique, unique configurations of these things. But the, the knowledge that you need, w- what I refer to as, like, the frameworks, for... You need for interpreting them, I don't think. I think that's, those are finite. Um-

Lex Fridman 35:31 You think the frameworks are more important than the bulk of the knowl- so, like, framing

David Ferrucci 35:37 Yeah. ... describes the- Because what the frameworks do is they give you now the ability to interpret and reason, and to interpret and reason and to interpret and reason over the specifics in ways that other humans would understand.

Lex Fridman 35:49 What about the specifics? You know-

David Ferrucci 35:51 Well, you acquire the specifics by reading and by talking to other people.

Lex Fridman 35:55 So I'm mostly actually just even... If we can focus on even the beginning, the common sense stuff, the stuff that doesn't even require reading or it almost s- requires playing around with the world or something. Just being able to sort of manipulate objects, drink w- water and so on.

David Ferrucci 36:12 Right.

Lex Fridman 36:12 All of that. Every time we try to do that kind of thing in robotics or AI, it seems to be like an onion. (laughs) You seem to realize how much knowledge is really required to perform even some of these basic tasks. Do you have that sense as well? And if so, how do we get all those details? Are they written down somewhere? Do they have to be learned through experience?

David Ferrucci 36:39 So I think when, like if you're talking about sort of the physics, the basic physics around us, for example, acquiring information about... Acquiring how that works, um, yeah, man, I think that, I think there's a combination of things going... I think there's a combination of things going on. I think there is like fundamental pattern matching, like what we were talking about before, where you see enough examples, enough data about something, you just start assuming that. And with similar input, I'm gonna predict similar outputs. You don't, can't necessarily explain it at all. Um, you may learn very quickly that when you let something go, it falls to the ground.

Lex Fridman 37:16 That's a, that's a, such a-

David Ferrucci 37:17 But you can't necessarily explain that.

Lex Fridman 37:19 But that's such a deep idea, that if you let something go, like the idea of gravity.

David Ferrucci 37:26 Is it? I mean-

Lex Fridman 37:26 Then-

David Ferrucci 37:26 ... people were letting things go and counting on them falling well before they understood gravity.

Lex Fridman 37:30 But that seems to be a... That's exactly what I mean, is before you take a physics class or, uh, or study anything about Newton, just the idea that stuff falls to the ground and then be- be able to generalize that oth- all kinds of stuff falls to the ground, uh, it just seems like a non... If, without encoding it, like hard coding it in, it seems like a difficult thing to pick up. It seems like you have to have a lot of different knowledge to be able to integrate that into the framework, sort of into everything else. So both know that stuff falls to the ground and start to reason about sociopolitical discourse. (laughs) So have both, like the very basic and the high level reasoning decision-making. I guess my question is how hard is this problem? (laughs) And sorry to linger on it because, again, and we'll get to it for sure, is what Watson with Jeopardy! did, is take on a problem that's much more constrained but has the same hugeness of scale, at least from the outsider's perspective. So I'm asking the general life question of to be able to be an intelligent being and reasoning in the, in the world about both gravity and politics, how hard is that problem?

David Ferrucci 38:54 So, I think it's solvable.

Lex Fridman 38:56 (laughs) That's... Okay. No, beautiful. So what about, what about time travel?

David Ferrucci 39:04 (laughs)

Lex Fridman 39:05 Okay. On that topic.

David Ferrucci 39:08 Not as-

Lex Fridman 39:08 I'm... (laughs) Same answer?

David Ferrucci 39:08 Not as convinced.

Lex Fridman 39:09 Not as convinced yet, okay.

David Ferrucci 39:11 No, I think... I think it is. I think it is solvable. I mean, I think that it's a learn... It's first of all, it's about getting machines to learn. Learning is fundamental, and I think we're at, ready in a place that we understand, for example, how machines can learn in va- various ways. Right now, our learning s- our learning stuff is sort of primitive, in that we haven't sort of taught machines to learn the frameworks. We don't communicate our frameworks because of how shared they... In some cases we do, but, um, we don't annotate, if you will, all the data in the world with the frameworks that are inherent or underlying our understanding. Instead we just operate with the data. So if we wanna be able to reason over the data in similar terms, in the common frameworks, we need to be able to teach the computer, or at least we need to program the computer to acquire, to have access to and acquire, learn the frameworks as well and connect the frameworks to the data. I think this, I think this can be done. I think we can start... I think machine learning, for example, with enough examples, uh, can start to learn the- these basic dynamics. Will they relate them necessarily to gravity? Not unless they can also acquire those theories as well and put the experiential knowledge and connect it back to the theoretical knowledge. I think if we think in terms of these class of architectures that are, are designed to both learn the specifics, find the patterns, but also acquire the frameworks and connect the data to the frameworks, if we think in terms of robust architectures like this, I think there is a path toward getting there.

Lex Fridman 41:02 Do you think... In terms of encoding architectures like that, do you think systems that are able to do this will look like neural networks or representing... If you look back to the '80s and '90s with the expert systems, so more like graphs, uh, um, systems that are based in logic, able to contain a large amount of knowledge, where the challenge was the automated acquisition of that knowledge. So I guess the question is when you collect both the frameworks and the knowledge from the data, what do you think that thing will look like?

David Ferrucci 41:37 Yeah. So I mean, I think the thing... Asking the question they look like neural networks is a bit of a red herring. I mean, I think that they, they will, they will certainly do inductive or pattern match based reasoning and I've already experimented with architectures that combine both, that use machine learning and neural networks to learn certain classes of knowledge. In other words, to find repeated patterns in order, or in order for it to make good inductive guesses, but then ultimately to try to take those learnings and, and marry them. In other words, um, connect them to frameworks so that it can then reason over that, in terms other humans understand. So for example, at Elemental Cognition we do both. We have architectures that, that do both. But... Both those things, but also have a learning method for acquiring the frameworks themselves and saying, "Look, ultimately I need to take this data. I need to interpret it in the form of these frameworks so they can reason over it." So there is a fundamental knowledge representation like what you were saying, like these graphs of logic, if you will. There are also neural networks that acquire certain class of information. Uh, they then, they then align them with these frameworks-... but there's also a mechanism to acquire the frameworks themselves.

Lex Fridman 42:49 Yeah, so it seems like the idea of frameworks requires some kind of collaboration with humans.

David Ferrucci 42:55 Absolutely.

Lex Fridman 42:56 So, do you think of that collaboration as-

David Ferrucci 42:59 Well, and let's to be clear, let's be clear. Only for the, for the, the expressed purpose that you're designing, you're, you, you're designing machi... You're designing an intelligence that can ultimately communicate with humans-

Lex Fridman 43:12 Mm.

David Ferrucci 43:12 ... in ter- in the terms of frameworks that help them understand things, right? So, so now to be really clear, you can create... You can independently create an a, a, a machine learning system and an intel... an intelligence that I might call an alien intelligence that does a better job than you with some things, but can't explain the framework to you. That doesn't mean it isn't... it might be better than you at the thing. It might be that you cannot comprehend the framework that it may have created for itself that is inexplicable to you. That's a reality.

Lex Fridman 43:45 But you're, you're more interested in, in a case where you can.

David Ferrucci 43:49 I, I am. I am, yeah. Uh, I p- My, my sort of approach to AI is because I've set the goal for myself, I want machines to be able to ultimately communicate, um, understanding with human. I want them to be able to acquire and communicate, acquire knowledge from humans and communicate knowledge to humans. They should be using what, you know, inductive, uh, machine learning techniques are good at, which is to observe patterns of data, whether it be in language or whether it be in images or mo- videos or whatever, to acquire these patterns, to induce the generalizations from those patterns, but then ultimately work with humans to connect them to frameworks, interpretations if you will, that ultimately make sense to humans. Of course, the machine is gonna have the strength that it has, the richer and longer memory, but that, you know, it has the more rigorous re- reasoning abilities, the deeper reasoning abilities. So it'd be an interesting, you know, complementary relationship between the human and the machine.

Lex Fridman 44:53 Do you think that ultimately needs explainability like a machine? So if you look, uh, you study, for example, Tesla autopilot a lot, or humans, I don't know if you've driven the vehicle or, are, are, are aware of what-

David Ferrucci 45:03 Yeah.

Lex Fridman 45:04 ... the, the... So you basically, the human and machine are working together there, and the human is responsible for their own life to monitor the system. And, you know, the system fails every few miles. And so the- there's, there's hundreds of... there's millions of those failures a day, and so that's like a moment of interaction. Do you see?

David Ferrucci 45:26 Yeah. That- that- that- no, that's exactly right. That's a moment of interaction, um, where, you know, the, the, the machine has learned some stuff, uh, it l- has a failure. Somehow the failure is communicated. The human is now filling in the, the mistake, if you will, or maybe correcting it or doing something that is more successful. In that case, the computer takes that learning. So I believe that the collaboration between human and machine, I mean, that's sort of a primitive example and sort of a more, um... Another example is where the machine is literally talking to you and saying, "Look, I'm, I'm reading this thing. I know, I know that, like, the next word might be this or that, but I don't really understand why. I have my guess. Can you help me understand the framework that supports this?" And then can kind of t- acquire that, take that, and reason about it and reuse it the next time it's reading to try to understand something. Not un- not unlike a human, uh, student might do. I mean, I remember like, uh, when my, my daughter was in first grade and she was... had a, um, re- reading assignment about electricity and, you know, somewhere in, in, in the text it says, "And electricity is produced by water flowing over turbines," or something like that. And then there's a question that says, "Well, how is electricity created?" And so my daughter comes to me and says, "I mean, I could..." You know, created and produced are kind of synonyms in this case. "So I can go back to the text and I can copy by water flowing over turbines, but I have no idea what that means."

Lex Fridman 46:55 Mm-hmm.

David Ferrucci 46:56 "Like, I don't know how to interpret water flowing over turbines and what electricity even is. I mean, I can get the answer right by matching the text, but I don't have any framework for understanding what this means at all."

Lex Fridman 47:07 And framework really is... I mean, it's a set of, not to be mathematical, but axioms of ideas that you bring to the table in interpreting stuff and then you build those up somehow.

David Ferrucci 47:18 You, you, you build them up with the expectation that there's a shared understanding of what they are.

Lex Fridman 47:23 Share... Yeah, yeah. It's the social-

David Ferrucci 47:25 Yeah, right.

Lex Fridman 47:26 ... that, that us humans... Do you have a sense that humans on Earth, in general, share a set of f... Like, how many frameworks are there?

David Ferrucci 47:36 I mean, it depends on how you bound them, right? So, in other words, how big or small, like their, their individual scope. Um, but there's lots and there are new ones. I think they're... I, I think the way I think about is kind of in a layer. I think of the architecture as being layered in that there's, there's a small set of primitives that allow you the foundation to build frameworks and then there may be, you know, many frameworks, but you have the ability to acquire them, and then you have the ability to reuse them. I mean, one of the most compelling ways of thinking about this is little reasoning by analogy where I can say, "Oh, wow, I've learned something very similar." Um, you know, I never heard of this, I never heard of this game, uh, soccer, but, um, if it's like basketball in the sense that the goal is like the hoop and I have to get the ball in the hoop and I have guards and I have this and I have that, like where, where does the... where, where are the similarities and where are the differences? And I have a foundation now for interpreting this new information.

Lex Fridman 48:31 And then, uh, different groups, like the millennials will have a framework and then, and then, and then-

David Ferrucci 48:37 Well, that, you know that-

Lex Fridman 48:37 ... and then ever-

David Ferrucci 48:38 Yeah.

Lex Fridman 48:38 You know?

David Ferrucci 48:39 Well, like that, that-

Lex Fridman 48:39 The Democrats and Republicans.

David Ferrucci 48:41 Well-

Lex Fridman 48:41 Millennials, nobody wants that framework, I think.

David Ferrucci 48:43 Well, well, I mean, I think-

Lex Fridman 48:44 No one understands it.

David Ferrucci 48:45 Right. I mean, they're talking about political and social ways of interpreting the world around them, and I think these frameworks are still largely, largely similar. I think they differ in maybe what some fundamental-

Lex Fridman 48:55 Yeah.

David Ferrucci 48:55 ... assumptions and values are. Now-... from a reasoning perspective, like the ability to process the framework of mine- m- might not be that different. The implications of different fundamental values or fundamental assumptions in those framework- frameworks may reach very different conclusions. So, from-

Lex Fridman 49:12 Mm-hmm.

David Ferrucci 49:13 ... so from a- a social perspective, the conclusions may be very different. From an intelligence perspective, I've- I, you know, I just followed where my assumptions took me.

Lex Fridman 49:21 Yeah, the pro- the process itself will look similar. But that's a fascinating idea that, um, frameworks really help carve how, uh, a statement will be interpreted.

David Ferrucci 49:33 Mm-hmm.

Lex Fridman 49:33 I mean, to having a- a- a- a- a Democrat and a Republican framework (laughs) and then read the exact same statement, and the conclusions that you derive will be totally different from an AI perspective is fascinating.

David Ferrucci 49:47 What we would want out of the AI is to be able to tell you that this perspecti- one perspective, one set of assumptions is gonna lead you here, another set of assumptions-

Lex Fridman 49:56 Yeah, absolutely.

David Ferrucci 49:56 ... is gonna leave you there. And to e- and in fact, you know, to help people reason and say, "Oh, I see where, I see where our differences lie."

Lex Fridman 50:04 Yeah.

David Ferrucci 50:05 You know, I have this fundamental belief about that. I have this fundamental belief about that.

Lex Fridman 50:09 Yeah, it's quite brilliant. From my perspective, an- uh, NLP, there's this idea that there's one way to really understand a statement, but th- there probably isn't. There's probably an infinite number of ways to understand a statement-

David Ferrucci 50:20 There's a lot... Well, there's lots-

Lex Fridman 50:21 ... depending on the framework.

David Ferrucci 50:22 ... there's lots of different interpretations. And the- the, you know, the- the broader, you know, the broader the- the content, the richer it is. Uh, and so, you know, you- you and I can have very different experiences with the same text, obviously. And if we're committed to understanding each other, we start... And that's the other important point. Like, if we're committed to understanding each other, we start decomposing and breaking down our interpretation towards more and more primitive components.

Lex Fridman 50:53 Mm-hmm.

David Ferrucci 50:54 Until we get to that point where we say, "Oh, I see why we disagree," and we try to understand how fundamental that disagreement really is. But that requires a commitment to breaking down that interpretation in terms of that framework in a logical way. Otherwise, you know, and this is why like I-I think of AI as- as really complementing and helping human intelligence to overcome some of its biases and its predisposition to be persuaded by, uh, you know, by this bu- but more shallow reasoning in the sense that like we get over this idea, well, I, you know- you know, I'm right because I'm a Republican or I'm right because I'm a Democratic, and someone labeled this as Democratic point of view, or it has the following keywords in it.

Lex Fridman 51:35 Mm-hmm.

David Ferrucci 51:35 And if- and- and if the machine can help us break that argument down and say, "Wait a second, you know, wh- what do you really think about this," right? So, essentially holding us accountable to doing more critical thinking.

Lex Fridman 51:47 We're gonna have to s- sit and think about this fast. That's- I love that. I- I think that's really empowering use of AI for the public discourse that's completely, uh, disintegrating currently on, uh-

David Ferrucci 51:58 Right.

Lex Fridman 51:58 ... as we learn how to do it on social media.

David Ferrucci 52:00 That's right.

Lex Fridman 52:02 So, one of the greatest accomplishments in the history of AI is, um, Watson competing against Je- uh, in- in a game of Jeopardy against humans, and you were a lead in that, a crit- a critical part of that. So, let's start at the very basics. What is the game of Jeopardy? The game for us humans, human versus human.

David Ferrucci 52:25 Right. So it's to take a question and answer it. (laughs)

Lex Fridman 52:32 (laughs)

David Ferrucci 52:33 Um, the game of Jeopardy! Well-

Lex Fridman 52:34 It's just the opposite.

David Ferrucci 52:35 ... actually, it's- actually-

Lex Fridman 52:36 (laughs) It's the opposite.

David Ferrucci 52:36 Well, well, no, but it's not, right?

Lex Fridman 52:38 Right.

David Ferrucci 52:38 (laughs) It's like- it's really not. It's really-

Lex Fridman 52:39 Yeah.

David Ferrucci 52:40 ... it's really to get a question and answer, but it's- it's what we call a factoid question. So this notion of like it's- it really relates to some fact that every- few people would argue whether the facts are true or not. In fact, most people would. And Jeopardy! kind of counts on the idea that these- these statements have factual answers. And, um, and the idea is to, first of all, determine whether or not you know the answer, which is sort of an interesting twist.

Lex Fridman 53:06 So, first of all, understand the question, right?

David Ferrucci 53:07 You have to understand the question. What is it asking? And that's a good point because the questions are not asked directly, right? They're-

Lex Fridman 53:14 They're all like... The way the questions are asked is non-linear. It's like, uh, it's a little bit witty. It's a little bit playful sometimes. It's, uh, it's a little bit tricky.

David Ferrucci 53:26 Yeah, they're asked in- in exactly numerous witty, tricky ways.

Lex Fridman 53:29 Yeah.

David Ferrucci 53:30 Uh, exactly what they're asking is not obvious. It takes- it takes inexperienced humans a while to go, "What is it even asking?"

Lex Fridman 53:36 Right.

David Ferrucci 53:37 And it's sort of an interesting realization that you have when somebody says, "Oh, what's the... Jeopardy! is a question answering show," and then he's like, "Oh, like I know a lot," and then you read it and you're- you're still trying to process the question, and the champions have answered and moved on. They're three like-

Lex Fridman 53:49 (laughs) Yeah.

David Ferrucci 53:50 ... they're three questions ahead by the (laughs) by the time you figured out what the question even meant. So, there's- there's definitely an ability there to just parse out what the question even is.

Lex Fridman 53:59 Yeah.

David Ferrucci 53:59 So, that was certainly challenging. It's interesting historically though, if you look back at the Jeopardy games much earlier, you know-

Lex Fridman 54:06 Like 60s, 70s, that kind of thing?

David Ferrucci 54:08 ... early games, the questions were much more direct. They weren't quite like that. They got sort of more and more interesting. The way they asked them that sort of got more and more interesting and subtle and nuanced and humorous and witty over time which really required the human to kind of make the right connections in figuring out what the question was even asking. So yeah, you have to figure out what the question's even asking, then you have to determine whether or not you think you know the answer, and because you have to buzz in really quickly, you still have to make that determination, uh, as quickly as you possibly can. In other words, y- you lose the opportunity to buzz in. You may-

Lex Fridman 54:43 Even before you really know if you know the answer.

David Ferrucci 54:46 I think a lo- I think a lot of humans will- will assume. They'll- they'll- they'll look at- they'll look at it- they're processed very superficially. In other words, what's the topic? What are some keywords? And just say, "Do I know this area or not?" before they actually know the answer.... then they'll buzz in and, then they'll buzz in and think about it. So it's interesting what humans do. Now, some people who know all things, like Ken Jennings or something, or the more recent big Jeopardy! player, um, they, I mean they'll just buzz in. They'll just assume they know all of Jeopardy! and they'll just buzz in.

Lex Fridman 55:15 Hmm.

David Ferrucci 55:15 You know, Watson, interestingly, didn't even come close to knowing all of Jeopardy!, right? Wa- Watson really-

Lex Fridman 55:20 Even at the peak, even at its best.

David Ferrucci 55:22 Yeah. So for example, I mean, we had this thing called Recall, which is like how many... Of all the Jeopardy! questions, you know, how many did, could we even find, like, find the right answer for, like, anywhere? Like, could we come up with... If we, look, you know, we had a, a big body of knowledge, som- something on the order of several terabytes. I mean, from, from a web scale, it was actually very small, but from, like, a book scale, I'm talking about millions and, of books, right? So the equivalent of millions of books. Encyclopedias, dictionaries, books, so it's, still a ton of information. And, you know, for, I think it was 80, only 85% was the answer anywhere to be found.

Lex Fridman 55:57 Hmm.

David Ferrucci 55:57 So you're already down, you're already down at that level just to gets, just to get started, right? So, and, and so it was important to get a, a very quick sense of, do you think you know the right answer to this question? So we had to compute that confidence as quickly as we possibly could. So it's, in effect, we had to answer it. And at least, you know, spend some time essentially answering it, and then judging the confidence that we, you know, that, that our answer was right. And then deciding whether or not we were confident enough to buzz in. And that would depend on what else was going on in the game, be- because there was a risk. So, like, if you're really in a situation where I have to take a guess, I have very little to lose, then you'll buzz in with less confidence.

Lex Fridman 56:40 So that was a counter for the, uh, the financial standings of the different competitors.

David Ferrucci 56:44 Correct. You know, what, what, how much of the game was left, how much time was left, um, where you were in the standings, things like that.

Lex Fridman 56:50 What, how many hundreds of milliseconds that we're talking about here? Do you have a sense of what is...

David Ferrucci 56:56 We targeted-

Lex Fridman 56:56 Like if S cons- What's, what's the target?

David Ferrucci 56:58 Yeah. We targeted... So, I mean, we targeted answering in under three seconds. And-

Lex Fridman 57:04 Buzzing in. So the decision to buzz in, and then the actual answering, are those two different stages?

David Ferrucci 57:11 Yeah. They were two, they were two different things. In fact, we had multiple stages, whereas, like, we would say, "Let's estimate our confidence." Which, which was sort of a shallow answering process. And then ultimate- and then ultimately decide to buzz in, and then we may take another second or something to kind of go in there and, and, and, and, and do that.

Lex Fridman 57:30 Make sure.

David Ferrucci 57:30 But by and large, we're saying, like, "We can't play the game, we can't even compete if we can't, on average, answer these questions in around three seconds or less."

Lex Fridman 57:40 So you stepped in. So there's this, there's these three humans playing a game, and you stepped in with the idea that IBM Watson would be one of, replace one of the humans and compete against, uh, two. Can you tell the story of Watson taking on this game?

David Ferrucci 57:56 Sure.

Lex Fridman 57:57 This seems exceptionally difficult.

David Ferrucci 57:58 Yeah. So the story, uh, was that, um, it was a- it was coming up, I think, the 10-year anniversary of, uh, of Big Blue. Uh, not Big Blue.

Lex Fridman 58:08 Deep Blue.

David Ferrucci 58:08 Deep Blue. IBM wanted to do sort of another kind of really, you know, fun challenge, public challenge, that can bring attention to IBM Research and the kind of the cool stuff that we were doing. I had been working in, in AI at IBM for some time. Uh, I had a team doing, uh, what's called open domain factoid question answering, which is, you know, we're not gonna tell you what the questions are. We're not even gonna tell you what they're about. Uh, can you go off and, and get accurate answers to these questions? And, um, it was an area of AI research that I was involved in. And so it was a big pa- it was a very specific passion of mine. Language understanding had always, always been a passion of mine. One sort of narrow slice on whether or not you could do anything with language was this notion of open domain, meaning I could ask anything about anything. Factoid, meaning it essentially had an answer.

Lex Fridman 58:57 Mm-hmm.

David Ferrucci 58:58 And, and, you know, being able to do that accurately and quickly. So that was a research area that my team had already been in. And so completely independently, several, you know, IBM executives were like, "What are, what are we going to do? What's the next cool thing to do?" And Ken Jennings was on his winning, uh, streak. This was like, whatever it was, 2004, I think, was on his win- winning streak. And someone thought, "Hey, that would be really cool if, um-"

Lex Fridman 59:21 (laughs)

David Ferrucci 59:21 "... if the computer can play Jeopardy!" And so this was-

Lex Fridman 59:24 Yeah.

David Ferrucci 59:24 ... like in 2004, they were shopping this thing around. And everyone was telling the ex- the, the research execs, um, "No way. Like, this is crazy." And we have some pretty, you know, senior people in the field, and they're saying, "Yeah, this is crazy." And it'll come across my desk, and I was like, "But that's kind of what, what I'm really interested in doing." And, um, but there was such this prevailing sense of, "This is nuts. We're not going to risk IBM's reputation on this. We're just not doing it." And this happened in 2004. It happened in 2005. At the end of 2000, um, 6, um, it was coming around again. And I was coming off of a, um... I was doing the, the open domain question answering stuff, but I was coming off a co- couple other projects. I had a lot more time to put into this. And I argued that it could be done, and I argued it would be crazy not to do this.

Lex Fridman 1:00:12 Can I, um... You could be honest at this point. So even though you argued for it, uh, what's the confidence that you had yourself, uh, privately, uh, that this could be done? What was... (laughs) We just told, told the story, how you tell stories to convince others.

David Ferrucci 1:00:27 Mm-hmm.

Lex Fridman 1:00:27 How confident were you? What was your estimation of the problem at that time?

David Ferrucci 1:00:32 So I thought it was possible, and a lot of people thought it was impossible. I thought it was possible.

Lex Fridman 1:00:37 Okay.

David Ferrucci 1:00:37 The reason why I thought it was possible is because I did some brief experimentation. I knew a lot about how we were approaching o- open domain factoid question answering. We had, we had been doing it for some years. I looked at the Jeopardy! stuff. I said, "This is gonna be hard," for a lot of the, uh, points that you, we mentioned earlier. Hard to interpret the question, um, hard to do it quickly enough, hard to compute an accurate confidence. None of this stuff had been done well enough before. But a lot of the technologies we're building were the kinds of technologies that should work. But more to the point, what was driving me was, I was in IBM Research-I was a senior leader in IBM Research, and this is the kind of stuff we were supposed to do.

Lex Fridman 1:01:17 Yeah, yeah.

David Ferrucci 1:01:17 In other words, we were basically supposed to-

Lex Fridman 1:01:18 This is the moon shot. This is the-

David Ferrucci 1:01:20 I mean, we were supposed to take things and say, "This is an active research area. It's our obligation to kind of, if we have the opportunity, to push it to the limits, and if it doesn't work, to understand more deeply why we can't do it." And so, I was very committed to that notion, saying, "Folks, this is what we do. It's crazy not, not to do it."

Lex Fridman 1:01:42 Yeah.

David Ferrucci 1:01:42 "This is an active research area. We've been in this for years. Why wouldn't we take this grand challenge and, and push it as hard as we can?" At the very least, we'd be able to come out and say, "Here's why this problem is, is way hard."

Lex Fridman 1:01:56 Yeah.

David Ferrucci 1:01:57 "Here's what we tried and here's how we failed." So, I was very driven, um, as a scientist from that perspective, and then I also argued, based on what we did a feasibility study, uh, why I thought it was hard but possible, and I showed examples of, you know, where it succeeded, where it failed, why it failed, and sort of a high level architectural approach for why we should do it. But for the most part, that... at that point, the execs really were just looking for someone crazy enough to say yes, because for s- several years at that point, everyone had said no.

Lex Fridman 1:02:29 Yeah.

David Ferrucci 1:02:29 "Um, I'm not willing to risk my reputation and my (laughs) career, you know, on this thing."

Lex Fridman 1:02:34 Clearly, you did not have such fears. Okay.

David Ferrucci 1:02:37 I, I did not.

Lex Fridman 1:02:38 So you dived right in, and yet, from what I understand, it was performing very poorly in the beginning. So, what were the initial approaches and why did they fail?

David Ferrucci 1:02:51 Well, there were lots of hard aspects to it. I mean, one of the reasons why prior approaches that we had worked on in the past, um, failed was because of... because the questions were difficult, difficult to interpret, like what are you even asking for, right?

Lex Fridman 1:03:09 Mm-hmm.

David Ferrucci 1:03:10 Very often, like if, if the question was very direct, like, "What city?" You know, or what, you know, even then it could be tricky, but, but you know, "What city?" Or, or, "What person?" Is often when it would name it very clearly, you would know that. And, and if there were just a small set of them, in other words, we're gonna ask about these five types. Like, it's gonna be an answer, and the answer will be, uh, a city in this state or a city in this country. The answer will be a person of this type, right? Like an actor or whatever it is. But it turns out that in Jeopardy! there were like tens of thousands of these things, and it was a very, very long tail. Meaning, you know, that it just went on and on, and, and so even if you focused on trying to encode the types at the very top, like, like there's five that were the most... let's say five of the most frequent, you'd still cover a very small percentage of the data. So you couldn't take that approach of saying, "I'm just going to try to collect facts about these five or 10 types or 20 types or 50 types or whatever." So, that was like one of the first things, like, what do you do about that? And so we came up with a, an approach toward that, and the approach looked promising, and we, we continued to improve our ability to, to, to handle that problem throughout the project. The other issue was that right from the outset I said, "We're not going to..." I committed to doing this in three to five years. So we did it in four, so I got lucky.

Lex Fridman 1:04:40 (laughs)

David Ferrucci 1:04:40 Um, but one of the things that that putting that like stake in the ground, was I kn- and I knew how hard the language understanding problem was. I said, "We're not going to actually understand language to solve this problem. We are not going to interpret the question and the domain of knowledge the question refers to, and reason over that to answer these questions. We're obvi- we're not gonna be doing that." At the same time, simple search wasn't good enough to, to confidently answer with a s- you know, a single correct answer.

Lex Fridman 1:05:13 First of all, that's like brilliant, that's such a great mix of innovation an- and practical engineering, three, three, four. Uh, so you're not, you're not trying to solve the general NL- NLU problem. You're saying, "Let's solve this in any way possible."

David Ferrucci 1:05:25 Oh, I, yeah, no.

Lex Fridman 1:05:26 But-

David Ferrucci 1:05:26 No, I was committed to s- saying, "Look, we're gonna solve an open domain question answering problem. We're using Jeopardy! as a driver for that."

Lex Fridman 1:05:33 As a, as a big benchmark.

David Ferrucci 1:05:34 Hard enough, big benchmark, exactly. And now, we're-

Lex Fridman 1:05:38 How do we do it?

David Ferrucci 1:05:38 We're just, like, whatever, like, just figure out what works, because I want to be able to go back to the academic scienc- c- community and say, "Here's what we tried. Here's what worked. Here's what didn't work."

Lex Fridman 1:05:47 This is great.

David Ferrucci 1:05:48 I don't want to go in and say, "Oh, I only have one techno- I have a hammer. I'm only gonna use this." I'm gonna do whatever it takes. I'm like, "I'm gonna think out of the box, I'm gonna do whatever it takes." One, um, and I also, there was another thing I believed. I believed that the fundamental NLP technologies and machine learning technologies would be, would be adequate. And this was an issue of how do we enhance them? How do we integrate them? How do we advance them? So I had one researcher who came to me, who had been working on question answering with me for a very long time, who had said, "We're gonna need Maxwell's equations for question answering." And I said, "If we, if we need some fundamental formula that breaks new ground in how we understand language, we're screwed."

Lex Fridman 1:06:32 Yeah.

David Ferrucci 1:06:33 "We're not gonna get there from here."

Lex Fridman 1:06:34 Yeah.

David Ferrucci 1:06:34 Like, we... I am not counting, I am... my assumption is I am not counting on some brand new invention. What I'm counting on is the ability to take everything that has done before, to figure out a, an architecture on how to integrate it well, and then see where it breaks, and make the necessary advances we need to make until this thing works.

Lex Fridman 1:06:58 Yeah. Push it hard to see where it breaks, and then-

David Ferrucci 1:07:01 Correct.

Lex Fridman 1:07:01 ... patch it up. I mean, that's how people change the world. I mean, that's the Elon Musk approach with rockets, SpaceX.

David Ferrucci 1:07:06 Right.

Lex Fridman 1:07:06 That's the, uh, Henry Ford and so on. I love it. But-

David Ferrucci 1:07:09 Right, and I, and I happened to be... and, and in this case, I happened to be right. But, but like we didn't know.

Lex Fridman 1:07:13 Right.

David Ferrucci 1:07:14 ... but you kind of have to put a stake in there because how are you going to run the project?

Lex Fridman 1:07:17 So, yeah, and backtracking to, uh, search. So if you were to do... What's the brute force solution? What, what would you search over? So you have a question, how would you search the possible space of answers?

David Ferrucci 1:07:31 Look, web search has come a long way even since then. Um, but at the time, like, you- you- you know, you... First of all, I mean, there are a couple other constraints around the problem, which is interesting. So you couldn't go out to the web. You couldn't search the internet. In other words, the AI experiment was we want a self-contained device. A device, if the device is as big as a room, fine, it's as big as a room, but we want a self-contained advice, uh, contained device. You're not going out to the internet. You don't have a lifeti- lifeline to anything. So it had to kind of fit in a shoebox, if you will, or at least a size of a few refrigerators, whatever it might be. So you... But also, you couldn't just get out there. You couldn't go off network, right, to, to kind of go. So there was that limitation. But then we did ex-... But the basic thing was go, go do, go do a web search. The problem was, even when we went and did a web search, I th- I don't remember exactly the numbers, but somewhere on the order of 65% of the time, the answer would be somewhere, you know, in the top 10 or 20 documents. So first of all, that's not even good enough to play Jeopardy!. Um, you, in other words, even if you could pull the, even if you could perfectly pull the answer out of the top 20 documents, top 10 documents, whatever it was, which we didn't know how to do, but even if you could th- do that, you're, you'd be... And, and you knew it was right and you had enough confidence in it, right? So you'd have to pull out the right answer. You have, you'd have to have confidence it was the right answer. And, and then you'd have to do that fast enough to now go buzz in, and you'd still only get 65% of them right-

Lex Fridman 1:09:00 Right.

David Ferrucci 1:09:00 ... which doesn't even put you in the winner's circle.

Lex Fridman 1:09:02 Right.

David Ferrucci 1:09:02 Winner's circle, you have to be up over 70, and you have to do it really qui- and you have to do it really quickly. But then now the problem is, well, even if I had somewhere in the top 10 documents, how do I figure out where in the top 10 documents that answer is and how do I compute a confidence of all the possible candidates? So it's not like I go in knowing the right answer-

Lex Fridman 1:09:21 Right.

David Ferrucci 1:09:21 ... and I have to pick it. I don't know the right answer. I have a bunch of documents. Somewhere in there is the right answer. How do I, as a machine, go out and figure out which one's right? And then how do I score it? So, and now how do I deal with the fact that I can't actually go out to the web?

Lex Fridman 1:09:37 First of all, if you pause on that, just think about it. If you could go to the web, do you think that problem is solvable? If you just pause on it, just thinking even beyond Jeopardy! Do you think the problem of reading text to find where the answer is?

David Ferrucci 1:09:53 Well, we solved, we solved that in some definition of solved, given the Jeopardy! challenge.

Lex Fridman 1:09:58 How did you do it for Jeopardy!? So how did you take a s- a body of work in a particular topic and extract the key pieces of information?

David Ferrucci 1:10:05 So, what... So, now forgetting about the, the, the huge volumes that are on the web, right? So now we, we have to figure out... We did a lot of source research. In other words, what body of knowledge is gonna be small enough but broad enough to answer Jeopardy!?

Lex Fridman 1:10:19 Yeah.

David Ferrucci 1:10:20 And we ultimately did find the body of knowledge that did that. I mean, it included Wikipedia and a bunch of other stuff.

Lex Fridman 1:10:25 So like encyclopedia type of stuff. I don't know if you can speak to-

David Ferrucci 1:10:27 Encyclopedia, dictionaries-

Lex Fridman 1:10:28 ... the different types of-

David Ferrucci 1:10:28 Different types of semantic resources, um, like WordNet and other types of semantic resources like that, as well as like some web crawls. In other words, where we went out and took that content and then expanded it based on producing statistical seed... you know, statistically producing seeds, using those seeds for other searchers, searches, and then expanding that. So using these, like, expansion techniques, we went out and had... found enough content that we're like, "Okay, this is good." And even up until the end, you know, we had a thread of researchers always trying to figure out what content could we efficiently include.

Lex Fridman 1:11:02 I mean, there's a lot of popular qu-... Like, "What is the church lady?" I think was one of the an-... Like-

David Ferrucci 1:11:07 Yeah.

Lex Fridman 1:11:07 ... what (laughs) w- where do you get... I guess you... That's probably an encyclopedia, so I guess-

David Ferrucci 1:11:12 So as-

Lex Fridman 1:11:12 ... the Beatles and some-

David Ferrucci 1:11:13 ... encyclopedia, but then, but then we would, but then we would take that stuff and we would go out and we would expand. In other words, we'd go find other content-

Lex Fridman 1:11:20 Right.

David Ferrucci 1:11:20 ... that wasn't in the core resources and expand it. You know, the amount of content, grew it by an order of magnitude. But still, so again, from a web scale perspective, this is very small amount of content.

Lex Fridman 1:11:30 It's very select. ?

Narrator 1:11:31 Very, very-

David Ferrucci 1:11:31 We then, we then took all that content. We, we pre-analyzed the crap out of it, meaning we, we, we parsed it, you know, broke it down into all those individual words, and we did semantic, uh, syntactic and semantic parses on it, you know, had computer algorithms that annotated it, and we inde- we indexed that in a very rich and very fast index. So we have a relatively huge amount of, you know, let's say the equivalent of, for the sake of argument, two to five million bucks. We've now analyzed all that, blowing up its size even more because now we have all this metadata, and we then we richly indexed all of that, and, and by the way, in a giant in-memory cache. So Watson did not go to disk.

Lex Fridman 1:12:12 So the infrastructure component there, if you could just speak to it, how tough, uh... I mean, I know 2000, maybe this is 2008, '9, you know, that, that's kind of a long time ago.

David Ferrucci 1:12:24 Right.

Lex Fridman 1:12:25 Uh, how hard is it to use multiple machines? Like, I mean, h- how hard is the infrastructure component, the hardware component?

David Ferrucci 1:12:31 So we used IBM, so we used IBM hardware. We had something like, uh, I forgot exactly, but 2,000 or close to 3,000 cores completely connected. So we had a switch where, you know, every CPU was connected to every other CPU.

Lex Fridman 1:12:43 And they were sharing memory in some kind of way.

David Ferrucci 1:12:46 Large-

Lex Fridman 1:12:46 Some kind of clever-

David Ferrucci 1:12:46 ... shared memory, right? And all this data was pre-analyzed and put into a very fast indexing structure that was-

Lex Fridman 1:12:55 That's awesome.

David Ferrucci 1:12:56 ... all, all, all in, all in memory. And then, um, we took that question... We would analyze the question. So all the content was now pre-analyzed. So if I... So if I went and tried to find a piece of content, it would come back with all the metadata that we had pre-computed.

Lex Fridman 1:13:14 How do you shove that question-How do you connect the, the big stuff with the meta, uh, the- the big knowledge base or the metadata and the- that's indexed to the- the simple little witty confusing question?

David Ferrucci 1:13:27 Right. So therein lies, you know, the Watson architecture, right? So we would take the question, we would analyze the question, so which means that we would parse it and interpret it a bunch of different ways. We'd try to figure out what is it asking about, so we would come ... we had multiple strategies to kind of determine what was it asking for that might be represented as a simple string, a character string, um, or something we would connect back to different semantic types that were from existing resources. So anyway, the bottom line is we would do a bunch of analysis on the question and question analysis had to finish and had to finish fast. Um, so we'd do the question analysis because then from the question analysis, we would now produce searches. So we would, um, and we had built, um, using open source search engines, we modified them, but we had a number of different search engines we would use that had different characteristics. We went in there and engineered and modified those search engines ultimately to now take our question analysis, produce multiple queries based on different interpretations, um, of the question and fire out a whole bunch of searches in parallel.

Lex Fridman 1:14:37 Oh.

David Ferrucci 1:14:37 And they would produ-

Lex Fridman 1:14:38 Very nice.

David Ferrucci 1:14:38 ... they would come back with passages. Um, so this is, these were passage search algorithms. They would come back with passages. And so now you, let's say you had a thousand passages, now for each passage you, you parallelize again. So you went out and you parallelized those, parallelized the search. Each search would now come back with a whole bunch of passages. Um, maybe you had a total of a thousand or 5,000 whatever passages. For each passage now you'd go and figure out whether or not there was a candidate, what we call a candidate answer in there. So you had a whole bunch of another ... a whole bunch of other algorithms that would find candidate answers, possible answers to the question. And so you had candidate answer gen- called candidate answers generators, a whole bunch of those. So for every one of these components the team was constantly doing research, coming up better ways to generate search queries from the questions, better ways to analyze the question, better ways to generate candidates.

Lex Fridman 1:15:31 And speed. So better is accuracy and, uh, speed.

David Ferrucci 1:15:35 Correct. So right, speed and accuracy for the most part were separated. We handled that sort of in separate ways, like I focus purely on accuracy, end-to-end accuracy. Are we ultimately getting more questions and producing more accurate confidences? And then I had a whole nother team that was constantly analyzing the workflow to find the bottlenecks and then figuring out how to both parallelize and drive the- the algorithm speed. But anyway, so- so now think of it like you have this big fan out now, right? Because you have ... you had multiple queries, now you have, now you have thousands of candidate answers. For each candidate answer, you're gonna score it. So you're gonna use all the data that built up. You're gonna use the question analysis.

Lex Fridman 1:16:15 Mm-hmm.

David Ferrucci 1:16:15 You're gonna use how the query was generated. You're gonna use the passage itself and you're gonna use the candidate answer that was generated, uh, and you're gonna score that. So now we have a group of researchers coming up with scorers. There are hundreds of different scorers.

Lex Fridman 1:16:31 Mm-hmm.

David Ferrucci 1:16:32 So now you're getting a fan added again from however many candidate answers you have to all the different scorers. So if you have a-a 200 different scorers and you have a thousand candidates, now you have two thou- ... hundred thousand scorers and-and so now you gotta figure out, you know, how do I now rank these, rank these answers based on the scorers that came back? And I want to rank them based on the likelihood that they're a correct answer to the question. So every scorer was its own research project.

Lex Fridman 1:17:01 What do you mean by scorer? So is that the annotation process of, of basically a human being saying that this, this answer is-

David Ferrucci 1:17:07 Yeah, think of it, think of it as-

Lex Fridman 1:17:08 ... has a quality of-

David Ferrucci 1:17:09 ... you can think of, think of it, if you want to think of it what you're doing, you know, if you want to think about what a human would be doing, a human would be looking at a possible answer. They'd be reading the, you know, Emily Dixon, Dickinson, they'd be reading the passage in which that occurred, they'd be looking at the question and they'd be making a decision of how likely it is that Emily Dixon, Dickinson, given this evidence in this passage, is the right answer to that question.

Lex Fridman 1:17:33 Ah, got it. So that, that's the annotation task. That's the annotation process.

David Ferrucci 1:17:36 That's the scoring task.

Lex Fridman 1:17:38 So but scoring implies zero to one kind of continuous-

David Ferrucci 1:17:41 That's right, you give it a zero to one score. (laughs)

Lex Fridman 1:17:43 So it's not a binary ...

David Ferrucci 1:17:44 No.

Lex Fridman 1:17:44 So it's-

David Ferrucci 1:17:45 You give it a score. You give it a zero to ... yeah, exactly, a zero to one score.

Lex Fridman 1:17:48 So but humans d- give different scores so you have to somehow normalize and all that kind of stuff, uh, deal with all that complexity.

David Ferrucci 1:17:54 Depends on what your strategy is. We both, we- we-

Lex Fridman 1:17:57 It could be relative too. It could be, uh ...

David Ferrucci 1:17:59 We- we actually looked at the raw scores as well as standardized scores because humans are not involved in this. Humans are not involved.

Lex Fridman 1:18:05 Sorry, so I'm- I'm misunderstanding the- the- the process here. This- this is passages. Where is the ground truth coming from?

David Ferrucci 1:18:13 Ground truth is only the answers to the questions.

Lex Fridman 1:18:16 So it's end-to-end.

David Ferrucci 1:18:17 It's end-to-end. So we al- ... so I was always driving end-to-end perform- ... it was a very interesting-

Lex Fridman 1:18:23 Wow.

David Ferrucci 1:18:23 ... a very interesting, you know, engineering, um, approach and ult- ultimately scientific and research approach were always driving end-to-end. Now that's not to say we- we- we wouldn't make hypotheses that individual component performance was related in some way to end-to-end performance.

Lex Fridman 1:18:44 Right.

David Ferrucci 1:18:44 Of course we would because people would have to build individual components. But ultimately to get your component integrated into the system, you had to show impact on end-to-end performance, question answering performance.

Lex Fridman 1:18:55 So there's- there's many very smart people working on this and they're basically trying to- to, uh, sell their ideas as a component that should be part of the system.

David Ferrucci 1:19:03 That's right. And- and they would do research on their component and they would say things like-

Lex Fridman 1:19:08 So cool.

David Ferrucci 1:19:09 ... you know, "I'm gonna improve this as a candidate generator," or, "I'm gonna improve this as a question scorer or as a passage scorer, I'm gonna improve this, or as a parser and I can improve it by 2%."... on its component metric, like a better parse or a better candidate, or a better type estimation, or whatever it is. And then I would say, "I need to understand how the improvement on that component metric is going to affect the end-to-end performance." If you can't estimate that and can't do experiments that demonstrate that, it doesn't get in.

Lex Fridman 1:19:43 That's like the best run AI project I've ever heard. That's awesome. Okay. Uh, what breakthrough would you say... Like, I'm sure there's a lot of day-to-day breakthroughs, but was there like a breakthrough that really helped improve performance? Like where, where people began to believe? (laughs) Or is it just a gradual process

David Ferrucci 1:20:02 Well, I