Ayanna Howard: Human-Robot Interaction & Ethics of Safety-Critical Systems | Lex Fridman Podcast #66

1. 0:00 – 2:09

   Introduction

   1. LFLex Fridman0:00

      The following is a conversation with Ayanna Howard. She's a roboticist, professor at Georgia Tech, and director of the Human Automation Systems Lab, with research interests in human-robot interaction, assistive robots in the home, therapy gaming apps, and remote robotic exploration of extreme environments. Like me, in her work, she cares a lot about both robots and human beings, and so I really enjoyed this conversation. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcast, follow on Spotify, support it on Patreon, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D-M-A-N. I recently started doing ads at the end of the introduction. I'll do one or two minutes after introducing the episode, and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends, but you can also use it to buy, sell, and deposit bitcoin in just seconds. Cash App also has a new investing feature. You can buy fractions of a stock, say $1 worth, no matter what the stock price is. Broker services are provided by Cash App Investing, a subsidiary of Square and member SIPC. I'm excited to be working with Cash App to support one of my favorite organizations called FIRST, best known for their FIRST robotics and Lego competitions. They educate and inspire hundreds of thousands of students in over 110 countries, and have a perfect rating on Charity Navigator, which means the donated money is used to maximum effectiveness. When you get Cash App from the App Store or Google Play, and use code LEXPODCAST, you'll get $10, and Cash App will also donate $10 to FIRST, which again is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now, here's my conversation with Ayanna Howard.

2. 2:09 – 5:05

   Favorite robot

   1. LFLex Fridman2:09

      What, or who, is the most amazing robot you've ever met, or perhaps had the biggest impact on your career?

   2. AHAyanna Howard2:16

      I haven't met her, but I grew up with her, but of course Rosie. So, and I think it's because also-

   3. LFLex Fridman2:25

      Who's Rosie?

   4. AHAyanna Howard2:26

      Rosie from the Jetsons. She is all things to all people, right?

   5. LFLex Fridman2:31

      (laughs)

   6. AHAyanna Howard2:31

      Think about it, like anything you wanted, it was like magic, it happened. Um-

   7. LFLex Fridman2:35

      So people not only an-anthropomorphize, but project whatever they wish for the robot to be onto-

   8. AHAyanna Howard2:41

      Onto Rosie.

   9. LFLex Fridman2:42

      ... Rosie.

   10. AHAyanna Howard2:43

       But also, I mean, think about it, she was socially engaging. She every so often had an attitude, right? Um, she kept us honest. She, she would push back sometimes when, you know, George was doing some weird stuff. Um, but she cared about people, especially the kids. Uh, she's, she was like the, the perfect robot. (laughs)

   11. LFLex Fridman3:04

       And, and you've said that people don't want their robots to be perfect. Can you elaborate that? What do you think that is? Just like you said, uh, Rosie pushed back a little bit every once in a while.

   12. AHAyanna Howard3:15

       Yeah. So I, I think it's that... So if you think about robotics in general, we want them because they enhance our quality of life, and, and usually that's linked to something that's functional, right? Uh, even if you think of self-driving cars, why is there a fascination? Because people really do hate to drive. Like there's the, like Saturday driving where I can just speed, but then there's the I have to go to work every day and I'm in traffic for an hour. I mean, people really hate that. Um, and so robots are designed to basically enhance our ability to increase our quality of life, and so the perfection comes from this aspect of interaction. Um, if I think about how we drive, if we drove perfectly, we would never get anywhere, right? So think about how many times you had to, uh, run past the light because you see the car behind you is about to crash into you, or that little kid, uh, kind of runs into the, the street and so you have to cross on the other side 'cause there's no cars, right?

   13. LFLex Fridman4:18

       Mm-hmm.

   14. AHAyanna Howard4:18

       Like if you think about it, we are not perfect drivers. Some of it is because it's our world, and so if you have a robot that is perfect in that sense of the word, they, they wouldn't really be able to function with us.

   15. LFLex Fridman4:31

       Can you linger a little bit on the word perfection? Sort of from the robotics perspective, what does that word mean and how is sort of the optimal behavior, as you're describing, different than the, what we think of as perfection?

   16. AHAyanna Howard4:46

       Yeah. So perfection, if you think about it in the more theoretical point of view, it's really tied to accuracy, right? So if I have a function, can I complete it at 100% accuracy with zero errors? Um, and so that's kind of if you think about perfection, um, in the sense of the word.

3. 5:05 – 8:43

   Autonomous vehicles

   1. AHAyanna Howard5:05

   2. LFLex Fridman5:05

      And in a self-driving car realm, do you think from a robotics perspective, we kind of think that, uh, perfection means following the rules perfectly, sort of defining staying in the lane, changing lanes, when there's a green light you go, when there's a red light you stop, and that, that's the f-... And be able to perfectly see all the entities in the scene? That's the limit of what we think of as perfection?

   3. AHAyanna Howard5:32

      A- and I think that's where the problem comes is that, uh, when people think about perfection for robotics, the ones that are the most successful are the ones that are, quote-unquote, "perfect." Like I said, Rosie is perfect, but she actually wasn't perfect in terms of accuracy, but she was perfect in terms of how she interacted and how she adapted. And I think that's some of the disconnect is that we really want perfection with respect to its ability to adapt to us. We don't really want perfection with respect to 100% accuracy with respect to the rules that we just made up anyway, right? And so I think there's this disconnect sometimes between what we really want...... and what happens. And we see this all the time, like in, in my research, right? Like the, the optimal, quote-unquote, "optimal interactions" are when the robot is adapting based on the person, not 100% following what's optimal based on the rules.

   4. LFLex Fridman6:29

      Just to linger on autonomous vehicles for a second, just your thoughts, maybe off the top of the head, uh, is how hard is that problem, do you think, based on what we just talked about? You know, there's a lot of folks in the automotive industry that are very confident, from Elon Musk to Waymo to, to all these companies. How hard is it to solve that last piece-

   5. AHAyanna Howard6:50

      The last mile. (laughs)

   6. LFLex Fridman6:51

      ... that's the, the, the gap between the perfection and the human definition of how you actually function in this world?

   7. AHAyanna Howard6:59

      Yeah, so this is a moving target. So I remember when, um, all the big companies started to heavily invest in this, and there was, uh, a number of even roboticists as well as, you know, folks who were putting in the VCs and, and corporations, Elon Musk being one of them, that said, you know, "Self-driving cars on the road with people, you know, within five years."

   8. LFLex Fridman7:20

      Mm-hmm.

   9. AHAyanna Howard7:21

      Uh, uh, that was a little while ago, um, and now people are saying, "Five years, 10 years, 20 years." Some are saying, "Never," right?

   10. LFLex Fridman7:31

       Mm-hmm.

   11. AHAyanna Howard7:31

       I think if you look at some of the things that are being successful is these, um, basically fixed environments where you still have some anomalies, right? You still have people walking, you still have stores, but you don't have, um, other drivers, right, like other human drivers, or it's a dedicated space for the, for the cars. Um, because if you think about robotics in general, where it's always been successful is in f-... I mean, you can say manufacturing, like way back in the day, right? It was a fixed environment. Humans were not part of the equation. We're a lot better than that, but, like when we can carve out scenarios that are closer to that space, then I think that it's where we are.

   12. LFLex Fridman8:16

       Mm-hmm.

   13. AHAyanna Howard8:17

       So a closed campus where you don't have self-driving cars-

   14. LFLex Fridman8:20

       Yep.

   15. AHAyanna Howard8:20

       ... and maybe some protection so that the students don't jet in front just because they wanna see what happens. Like having a little bit, I think that's where we're gonna see the most success in the near future.

   16. LFLex Fridman8:32

       And be slow-moving.

   17. AHAyanna Howard8:33

       Right. Not, not, you know, 55, 60, 70 miles an hour, but the, the speed of, uh, a golf cart, right? (laughs)

   18. LFLex Fridman8:42

       So that said,

4. 8:43 – 20:03

   Tesla Autopilot

   1. LFLex Fridman8:43

      the most successful, in the automotive industry, robots operating today in the hands of real people are ones that are traveling over 55 miles an hour and in unconstrained environments, which is Tesla vehicles, so the Tesla autopilot. So I just... I would love to hear sort of your just thoughts of, uh, two things. So one, I don't know if you've gotten to see or have heard about something called Smart Summon, uh, where Tesla system, autopilot system, where the car drives zero-occupancy, no driver in the parking lot, slowly sort of tries to navigate the parking lot to find itself to you. And there's some incredible amounts of videos and just hilarity that happens as it awkwardly tries to navigate this (laughs) environment. But it's-

   2. AHAyanna Howard9:31

      (laughs)

   3. LFLex Fridman9:31

      ... it's a beautiful nonverbal communication between machine and human that I think is, uh, from... i- it's like, it's some of the work that you do in this kind of interesting human/robot interaction space. So what are your thoughts in general about it?

   4. AHAyanna Howard9:43

      So I, I do have that feature.

   5. LFLex Fridman9:46

      Mm-hmm.

   6. AHAyanna Howard9:46

      Um...

   7. LFLex Fridman9:47

      Do you drive a Tesla?

   8. AHAyanna Howard9:47

      I do.

   9. LFLex Fridman9:48

      Oh, nice.

   10. AHAyanna Howard9:48

       Um, mainly because I'm a gadget freak, right?

   11. LFLex Fridman9:52

       (laughs)

   12. AHAyanna Howard9:52

       So I, I see... It's a, it's a gadget that happens to have some wheels.

   13. LFLex Fridman9:55

       Yeah.

   14. AHAyanna Howard9:55

       And yeah, I've seen some of the videos. Um...

   15. LFLex Fridman9:58

       But what's your experience like? I mean, you're, you're a human/robot interaction roboticist. You're a legit sort of expert in the field, so what does it feel for a machine to come to you?

   16. AHAyanna Howard10:08

       I- it's one of these very fascinating things, but also I am hyper, hyperalert, right? Like I'm hyper-alert.

   17. LFLex Fridman10:17

       (laughs) Yes.

   18. AHAyanna Howard10:17

       Like my butt... my thumb is like, "Oh, okay. I'm, I'm ready to take over." Um, even when I'm in my car or I'm doing things like automated backing into... Uh, so there's like a feature where you can do this automating backing into-

   19. LFLex Fridman10:31

       Mm-hmm.

   20. AHAyanna Howard10:31

       ... a parking space, um, or bring the car out of your garage, um, or even, you know, pseudo autopilot on the freeway.

   21. LFLex Fridman10:40

       Right.

   22. AHAyanna Howard10:40

       Right? I am hypersensitive. I can feel... Like as I'm navigating, I'm like, "Yeah, that's an error right there."

   23. LFLex Fridman10:46

       Yeah.

   24. AHAyanna Howard10:46

       Like I, I'm very aware of it, uh, but I'm also fascinated by it, and it does get better. Like it... I, I look and see it's learning from all of these people who are cutting it on.

   25. LFLex Fridman11:00

       Yeah.

   26. AHAyanna Howard11:00

       Like every time I cut it on-

   27. LFLex Fridman11:02

       It gets better and better.

   28. AHAyanna Howard11:03

       ... it's getting better, right? And so I think that's what's amazing about it is that.

   29. LFLex Fridman11:07

       So this nice dance of you're still hypervigilant, so you're still not trusting it at all- (laughs)

   30. AHAyanna Howard11:12

       Yeah, that's-
5. 20:03 – 28:11

   Ethical responsibility of safety-critical algorithms

   1. LFLex Fridman20:03

      w- do you think, from a robotics perspective, you know, if you, if you're kind of honest of what cars do, they, they kind of, we kind of threaten each other's life all the time. So cars are very as- I mean, in order to navigate intersections, there's an assertiveness, there's a risk-taking, and if you were to reduce it to an objective function, there's a probability of murder in that function, meaning you killing another human being, and you're using that... First of all, y- it has to be low enough to be acceptable to you on an ethical level, as a individual human being, but it has to be high enough for people to respect you, to not sort of take advantage of you completely, and jaywalk in front of you and so on. So h- uh, I mean, I don't think there's a right answer here, but what's, how do we solve that? How, how do we solve that from a robotics perspective when danger and human life is at stake?

   2. AHAyanna Howard21:00

      Yeah. As they say, cars don't kill people, people kill people.

   3. LFLex Fridman21:03

      People kill people. (laughs)

   4. AHAyanna Howard21:03

      Um, right. Um, so I, I think-

   5. LFLex Fridman21:08

      And now robotic algorithms would be killing people.

   6. AHAyanna Howard21:10

      Right. So it will be, uh, robotics algorithms that are pro... No. It would be robotic algorithms don't kill people, developers of robotic algorithms-

   7. LFLex Fridman21:18

      Developers of-

   8. AHAyanna Howard21:18

      ... kill people. Right? I mean-

   9. LFLex Fridman21:20

      So-

   10. AHAyanna Howard21:20

       ... one of the things is people are still in the loop, and at least in the near and mid-term, I think people will still be in the loop, at some point, even if it's the developer. Like, we're not necessarily at the stage where, you know, robots are programming autonomous robots with different behaviors-

   11. LFLex Fridman21:38

       Yeah.

   12. AHAyanna Howard21:38

       ... quite yet. Um, not-

   13. LFLex Fridman21:40

       That's a scary notion. Sorry to interrupt. That a developer is, has some responsibility in, in a, in the death of a human being. That's a-

   14. AHAyanna Howard21:50

       I mean, I think that's why-

   15. LFLex Fridman21:53

       ... that's a heavy burden.

   16. AHAyanna Howard21:53

       ... th- the whole aspect of, of ethics in our community is so, so important, right? Like, because it's true. If, if, if you think about it, um, you can basically say, "I'm not going to work on weaponized AI," right? Like people can say, "That's not what I'm gonna do." But yet, you are programming algorithms that might be used in healthcare a- algorithms that might decide whether this person should get this medication or not, and they don't, and they die. You, okay, so w- that is your responsibility, right? And if you're not conscious and aware that you do have that power when you're coding and, and things like that, I think that's, that's, that's just not a good thing. Like, we need to think about this responsibility as we program robots and, and computing devices, um, much more than we are.

   17. LFLex Fridman22:44

       Yeah, so it's not an option to not think about ethics. I think it's a majority, I would say, of computer science, sort of, there, it's kind of a hot topic now, I think about bias and so on, but it's, and we'll talk about it, but usually it's kind of, you (laughs) , it's like a very particular group of people that work on that. And then people who do, like, robotics are like, "Well, I don't have to think about that, you know, there's other smart people thinking about it." It seems that everybody has to think about it. It's not, you can't escape the e- ethics, whether it's bias or just every aspect of ethics that has to do with human beings, yeah.

   18. AHAyanna Howard23:22

       Everyone. So think about, I'm gonna age myself, but I remember, uh, when we didn't have, like, testers, right? And so what did you do? As a developer, you had to test your own code, right? Like, you had to go through all the cases and figure it out, and you know, and then they realized that, you know, like, we probably need to have testing because we're not getting all the things. And so from there, what happens is, like, most developers, they do, you know, a little bit of testing, but it's usually like, "Okay, did my compiler bug out?"

   19. LFLex Fridman23:49

       Mm-hmm.

   20. AHAyanna Howard23:49

       "Let me look at the warnings. Okay, is that acceptable or not?" Right? Like, that's how you typically think about it as a developer, and you'll just assume that it's going to go to another process, and-

   21. LFLex Fridman23:59

       Yeah.

   22. AHAyanna Howard23:59

       ... they're gonna test it out. But I think we need to go back to those early days when, you know, you're a developer, you're developing. There should be like this, a, you know, "Okay, let me look at the ethical outcomes of this." Because there isn't a second, like, testing, ethical testers, right? It's you.

   23. LFLex Fridman24:17

       Mm-hmm.

   24. AHAyanna Howard24:17

       Um, we did it back in the early coding days. Um, I think that's where we are with respect to ethics. Like, let's go back to what was good practices, and only because we were just developing the field.

   25. LFLex Fridman24:31

       Yeah, and it's, uh, I mean, it's a really h- heavy burden. I've f- I've had to feel it recently in the last few months, but I think it's a, it's a good one to feel. Like, I've gotten the message, more than one, from people...You know, I've unfortunately gotten some attention recently, and s- I've gotten messages that say that I have blood on my hands because of working on semi-autonomous vehicles. The idea that you have semi-autonomy means people will become... will lose vigilance, and so on. And it's actually, be humans, as we describe. And because of that, because of this idea that we're creating automation, there'll be people be hurt because of it, and I think that's a beautiful thing. I mean, it's, you know, there's many nights where I wasn't able to sleep because of this notion, you know? You really do think about people that might die because of this technology. Of course, you could then start rationalizing and saying, "Well, you know what? 40,000 people die in the United States every year, and we're trying to ultimately try to save lives." But the reality is, your code you've written might kill somebody, and that's an important burden to carry with you as you design the code.

   26. AHAyanna Howard25:41

       I don't even think of it as a burden if we train this concept correctly from the beginning. And I use... And not to say that coding is like being a medical doctor, but think about it. Medical doctors, if y- they've been in situations where their patient didn't survive, right? Do they give up and go away? No. Every time they come in, they know that there might be a possibility that this patient might not survive. And so when they approach every decision, like, that's in their back of their head, and so why isn't that... we aren't teaching... And those are tools though, right?

   27. LFLex Fridman26:17

       Yeah.

   28. AHAyanna Howard26:17

       They're given some of the tools to address that so that they don't go crazy, but we don't give those tools so that it does feel like a burden versus something of, "I have a great gift and I can do great, awesome good, but with it comes great responsibility." I mean, that's what we teach in terms of if you think about the medical schools, right? Great gift, great responsibility. I think if we just change the messaging a little-

   29. LFLex Fridman26:41

       Mm-hmm.

   30. AHAyanna Howard26:42

       ... great gift, being a developer, great responsibility, and this is how you combine those.
6. 28:11 – 38:20

   Bias in robotics

   1. AHAyanna Howard28:12

   2. LFLex Fridman28:13

      On a slightly related note, uh, y- in a recent paper, The Ugly Truth About Ourselves and Our Robot Creations, you, you discuss, you highlight some biases that may affect the functioning of various robotic systems. Can you talk through, if you remember, examples of some?

   3. AHAyanna Howard28:30

      There's a lot of examples. I usually...

   4. LFLex Fridman28:31

      What is bias, first of all? I mean...

   5. AHAyanna Howard28:33

      Yeah, so bias is, um, this... And, and so bias, which is different than prejudice. So bias is that we all have these preconceived notions about particular, um, everything from particular groups for... to habits to, um, identity, right? So, we have these predispositions, and so when we address a problem, we look at a problem, we make a decision, those preconceived notions, uh, might affect our, our outputs or outcomes.

   6. LFLex Fridman29:02

      So, there the bias could be positive and negative, and then is prejudice the negative kind of bias?

   7. AHAyanna Howard29:07

      Prejudice is the negative, right? So, prejudice is that not only are you aware of your bias, but you are then take it and have a negative outcome-

   8. LFLex Fridman29:18

      Mm-hmm.

   9. AHAyanna Howard29:18

      ... even though you're aware, like...

   10. LFLex Fridman29:20

       And there could be gray areas too? I mean, that's-

   11. AHAyanna Howard29:23

       There's always gray areas.

   12. LFLex Fridman29:24

       That's the challenging aspect of all ethical questions, right?

   13. AHAyanna Howard29:27

       Right, right. So, I always like... Uh, so there's, there's a funny one, and in fact, I think it, it might be in the paper 'cause I think I talk about, uh, self-driving cars. But think about this. We, um... For teenagers, right, typically, we... insurance companies charge quite a bit of money if you have a teenage driver.

   14. LFLex Fridman29:46

       Mm-hmm.

   15. AHAyanna Howard29:46

       So, you could say that's an age bias-

   16. LFLex Fridman29:50

       Yeah.

   17. AHAyanna Howard29:50

       ... right?

   18. LFLex Fridman29:50

       Yeah.

   19. AHAyanna Howard29:50

       But no one will cl- I mean, parents will be grumpy, but no one really says that that's not fair, that it's their-

   20. LFLex Fridman29:58

       That's interesting. We don't, uh... That's right, that's right. It's, uh... Everybody in human factors and safety research al- almost, I mean, is quite ruthlessly critical of teenagers. (laughs) And we don't question, "Is that okay? Is that okay to be ageist in this kind of way?"

   21. AHAyanna Howard30:17

       It is, and it is ageist, right?

   22. LFLex Fridman30:18

       Yeah, yeah.

   23. AHAyanna Howard30:18

       It's definitely age-

   24. LFLex Fridman30:19

       There's no question about it. Yeah.

   25. AHAyanna Howard30:20

       And so, so th- these are, these... This is a gray area, right?

   26. LFLex Fridman30:24

       Yeah.

   27. AHAyanna Howard30:24

       'Cause you, um, y- you know that, you know, teenagers are more likely-

   28. LFLex Fridman30:29

       Mm-hmm.

   29. AHAyanna Howard30:29

       ... to be in accidents-

   30. LFLex Fridman30:31

       Yeah.
7. 38:20 – 40:35

   AI in politics and law

   1. LFLex Fridman38:20

      So speaking of this really nice notion that AI is maybe flawed but better than humans, so just made me think of it, one example of flawed humans is our political system. Do you think, or, uh, y- you said, uh, judicial as well, do you have a hope for AI sort of, uh, being elected for president? Or of, or running our Congress or being able to be a powerful representative of the people?

   2. AHAyanna Howard38:54

      So I mentioned, and I truly believe that this whole world of AI is in partnerships with people. And so what does that mean? I, I don't believe, or, and maybe I just don't, I don't believe that we should have an AI for president, but I do believe that a president should use AI as an advisor, right? Like, if you think about it, um, every president has a cabinet of individuals that have different expertise that they should listen to, right? Like, that's kind of what we do, and you put smart people with smart expertise around certain issues, and you listen. I don't see why AI can't function as one of those smart individuals giving input. So maybe there's an AI on healthcare, maybe there's an AI on education and, right? Like all of these things that a human is processing, right? I, I s- because at the end of the day, there's people that are human that are going to be at the end of the decision, and I don't think as a world, as a culture, as a society, that we would totally beli- and this is us, like this is some fallacy about us. But we need to see that leader, that person as human, um, and most people don't realize that, like, leaders have a whole lot of advice, right? When they say something, it's not that they woke up... well, usually, they don't wake up in the morning and be like, "I have a brilliant idea," right? It's usually a, "Okay, let me listen. I have a brilliant idea, but let me get a little bit of feedback on this, like, okay." And then it's a, "Yeah, that was an awesome idea," or it's like, "Yeah, let me go

8. 40:35 – 47:44

   Solutions to bias in algorithms

   1. AHAyanna Howard40:35

      back."

   2. LFLex Fridman40:36

      We already talked through a bunch of them, but are there some possible solutions to the biases present in our algorithms beyond what we just talked about?

   3. AHAyanna Howard40:46

      So I think there's two paths. One is to figure out how to systematically do the feedback and correction. So right now, it's ad hoc, right? It's a researcher identifies some outcomes that are not s- don't seem to be fair, right? They publish it. They write about it. Um, and they, either the developer or the companies that have adopted the algorithms may try to fix it, right? And so it's really ad hoc and it's not systematic. There's, it's just, it's kind of like, "I'm a researcher, that seems like an interesting problem," um, which means that there's a whole lot out there that's not being looked at, right? 'Cause it's kind of researcher driven. Um, I, and, and I don't necessarily have a solution, but that process I think could be done a little bit better. Uh, one way is, um, I'm going to poke a little bit at some of the corporations.

   4. LFLex Fridman41:47

      Mm-hmm.

   5. AHAyanna Howard41:47

      Right? Like, maybe the corporations, when they think about a product, they should, um, instead o- in addition to hiring these, you know, bug, they give these, um-

   6. LFLex Fridman41:59

      Oh, yeah, yeah, yeah. (laughs)

   7. AHAyanna Howard42:01

      ... where you, you-

   8. LFLex Fridman42:01

      Like awards when you find a bug.

   9. AHAyanna Howard42:03

      Yeah.

   10. LFLex Fridman42:03

       In your security, security hole.

   11. AHAyanna Howard42:04

       Yeah, a security bug.

   12. LFLex Fridman42:05

       Yeah.

   13. AHAyanna Howard42:05

       You know, let's, let's put it like, we will give the, whatever the award is that we give for the people who find these security holes, find an ethics hole, right?

   14. LFLex Fridman42:14

       Yeah.

   15. AHAyanna Howard42:14

       Like, find an unfairness hole and we will pay you X for each one you find. I mean, why can't they do that?

   16. LFLex Fridman42:19

       Yeah.

   17. AHAyanna Howard42:20

       One, it's a win-win. They show that they're concerned about it, that this is important, and they don't have to necessarily dedicate it their own, like, internal resources. And it also means that everyone who has, like, their own bias lens, like, "I'm interested in age and so I'll find the ones based on age," and, "I'm interested in gender and..." Right? Which means that you get, like, all of these different perspectives.

   18. LFLex Fridman42:41

       But you think of it in a data-driven way, so like we'll see... sort of, if we, if we look at a company like Twitter, it gets, it's under a lot of fire for discriminating against certain political beliefs.

   19. AHAyanna Howard42:54

       Correct.

   20. LFLex Fridman42:55

       And sort of, there's a lot of people, this is the, the sad thing, 'cause I know how hard the problem is, and I know the Twitter folks are working really hard at it, even Facebook that everyone seems to hate are working really hard at this. Uh, you know, w- the kind of evidence that people bring is basically anecdotal evidence. "Well, me or my friend, all we said is X, and for that, we got banned." And, and that's kind of a discussion of saying, well, look, that's usually, first of all, the whole thing is taken out of context, so there, they present sort of anecdotal evidence. And how are you supposed to as a company, in a healthy way, have a discourse about what is and isn't ethical? What, how do we make algorithms ethical when people are just blowing everything? Like, they're outraged about a particular anecdotal evident, piece of evidence that's very difficult to sort of contextualize in the big data-driven way. Do, do you have a hope for companies like Twitter and Facebook?

   21. AHAyanna Howard43:55

       Yeah. Um, so I think there, there's a couple of things going on, right? Uh, first off, uh, the, remember this whole aspect of we are becoming, um, reliant on technology. We're also becoming reliant on, um, a lot of these, um, the, the apps and the s- resources that are provided, right? So some of it is kind of anger, like, "I need you," right?

   22. LFLex Fridman44:21

       Yeah.

   23. AHAyanna Howard44:21

       "And you're not working for me." Right?

   24. LFLex Fridman44:23

       Yeah. (laughs) "You're not working for me," right?

   25. AHAyanna Howard44:24

       Um, but I think it's-

   26. LFLex Fridman44:26

       (laughs)

   27. AHAyanna Howard44:26

       ... a- and so some of it, and I, and I wish that, um, there was a little bit of change of rethinking. So some of it is like, "Oh, we'll fix it in-house." No. That's like, okay, I'm a fox, and I'm going to watch these hens because I think it's a problem that foxes eat hens.

   28. LFLex Fridman44:44

       Yeah.

   29. AHAyanna Howard44:44

       No, right? Like, use, like be good citizens and say, "Look. We have a problem, and we are willing to open ourselves up for others to come in and look at it, and not try to fix it in-house." Because if you fix it in-house, there's conflict of interest. If I find something, I'm probably gonna wanna fix it and hopefully the media won't pick it up, right? And that then causes distrust because someone inside is gonna be mad at you and go out and talk about how, "Yeah, they canned the resume"-

   30. LFLex Fridman45:15

       Yeah.
9. 47:44 – 49:57

   HAL 9000

   1. AHAyanna Howard47:44

   2. LFLex Fridman47:45

      So y- you've, uh, started your career at NASA Jet Propulsion Laboratory, but before I'd ask you a few questions there, have you happened to have ever seen Space Odyssey, 2001 Space Odyssey? (laughs)

   3. AHAyanna Howard47:57

      Yes.

   4. LFLex Fridman47:58

      Okay. Do you think (laughs) , do you think HAL 9000... So we're talking about ethics, do you think HAL did the right thing by taking the priority of the mission over the lives of the astronauts? Do you think HAL is good or evil? Easy questions today.

   5. AHAyanna Howard48:17

      Yeah. HAL was misguided.

   6. LFLex Fridman48:21

      You're one of the people that would be in charge of an algorithm like HAL.

   7. AHAyanna Howard48:26

      Yeah.

   8. LFLex Fridman48:26

      So how would you do better?

   9. AHAyanna Howard48:28

      If you think about what happened was there was no fail-safe, right? So we, uh, perfection, right? Like, what is that? I'm gonna make something that I think is perfect, but if my assumptions are wrong, it'll be perfect based on the wrong assumptions, right? That's something that you don't know until you deploy and then you're like, "Oh, yeah. Messed up." But what that means is that when we design software, such as in Space Odyssey, uh, when we put things out, that there has to be a s- fail-safe. There has to be the ability that once it's out there, you know, we can grade it as an F, and it fails-... and it doesn't continue, right?

   10. LFLex Fridman49:12

       Mm-hmm.

   11. AHAyanna Howard49:12

       If there's some way that it can be brought in and, and removed and, and that's aspect, uh, because that's what happened with, with HAL. It was like assumptions were wrong. It was perfectly correct based on those assumptions-

   12. LFLex Fridman49:27

       Mm-hmm.

   13. AHAyanna Howard49:27

       ... and there was no way to change, change it, change the assumptions at all.

   14. LFLex Fridman49:34

       And the change to fall back would be to a human, so you ultimately think like humans should be, uh, you know, it's not turtles or AI all the way down. It's, at some point there's a human that actually makes a decision.

   15. AHAyanna Howard49:48

       I still think that an- and again, because I do human-robot interaction, I still think the human needs to be part of the equation at some point.

   16. LFLex Fridman49:56

       So what

10. 49:57 – 51:53

    Memories from working at NASA

    1. LFLex Fridman49:57

       ... Just looking back, what are some fascinating things in robotic space that NASA was working at the time? Or just in general, what, what have you gotten to play with and what are your memories from working at NASA?

    2. AHAyanna Howard50:10

       Yeah. So one of my first memories was they were working on a surgical robot system that could do eye surgery, right?

    3. LFLex Fridman50:21

       Wow.

    4. AHAyanna Howard50:21

       And this was back in, oh my gosh, it must have been, oh, maybe '92, '93, '94.

    5. LFLex Fridman50:30

       So it's like, almost like a remote operation of a robot?

    6. AHAyanna Howard50:32

       Yeah. It was, it was remote operation, and in fact I, you can even find some old tech reports on it. So think of it, you know, like now we have da Vinci-

    7. LFLex Fridman50:41

       Mm-hmm.

    8. AHAyanna Howard50:41

       ... right? Like think of it, but these were like the late '90s, right? And I remember going into the lab one day and I was like, "What's that?" Right? And of course it, it wasn't pretty, right? 'Cause the technology, but it was like functional, and you had this, this individual that could use version of haptics to actually do the surgery, and they had this mockup of a human face and like the eyeballs, and you can see this l- little drill.

    9. LFLex Fridman51:08

       Mm-hmm.

    10. AHAyanna Howard51:08

        And I was like, "Oh, that is so cool."

    11. LFLex Fridman51:11

        (laughs)

    12. AHAyanna Howard51:11

        Um, that one I vividly remember, uh, because it was so outside of my, like, possible thoughts of what could be done.

    13. LFLex Fridman51:20

        Just the kind of precision and the ... I mean, what, what's the most amazing of a thing like that?

    14. AHAyanna Howard51:26

        I think it was the precision. It was the kind of first time that I had physically seen-

    15. LFLex Fridman51:34

        Hmm.

    16. AHAyanna Howard51:34

        ... this robot machine human interface, right? Versus ... 'Cause y- y- manufacturing had been, you saw those kind of big robots, right? But this was like, oh, this is ... And a person. There's a person-

    17. LFLex Fridman51:48

        Yeah.

    18. AHAyanna Howard51:48

        ... and, and a robot like-

    19. LFLex Fridman51:49

        Yeah.

    20. AHAyanna Howard51:50

        ... in the same space.

    21. LFLex Fridman51:51

        So meeting them in person. I,

11. 51:53 – 54:27

    SpotMini and Bionic Woman

    1. LFLex Fridman51:53

       like for me, it was a magical moment that I can't ... It was life transforming that I recently met Spot Mini from Boston Dynamics.

    2. AHAyanna Howard52:00

       Oh, see.

    3. LFLex Fridman52:01

       I don't know why, but on a human-robot interaction, for some reason I realized how easy it is to anthropomorphize.

    4. AHAyanna Howard52:09

       (laughs)

    5. LFLex Fridman52:09

       And it was, I don't know, I, it was, um, it was almost like falling in love with this feeling of meeting ... And I've obviously seen these robots a lot on video and so on, but meeting in person, just having that one-on-one time-

    6. AHAyanna Howard52:21

       It's different.

    7. LFLex Fridman52:21

       ... is different. So do you ha- Have you had a robot like that in your life that was ... made you maybe fall in love with robotics? Sort of o- like meeting in person? Or was it *******?

    8. AHAyanna Howard52:32

       I mean, I, I mean, I, I loved robotics since-

    9. LFLex Fridman52:35

       From the beginning? (laughs)

    10. AHAyanna Howard52:35

        Yeah. So ... (laughs) I was a 12-year-old, like-

    11. LFLex Fridman52:38

        Yeah.

    12. AHAyanna Howard52:38

        ... "I'm gonna be a roboticist."

    13. LFLex Fridman52:39

        Yeah.

    14. AHAyanna Howard52:39

        Uh, actually it was, I called it cybernetics. But, so my, my motivation was Bionic Woman. I don't know if you know that.

    15. LFLex Fridman52:45

        Yes.

    16. AHAyanna Howard52:45

        Um, and so, I mean, that was like a seminal moment, but I didn't meet ... Like, that was TV, right? Like, it wasn't like I was in the same space and I met and I was like, "Oh my gosh, you're, like, real."

    17. LFLex Fridman52:56

        Just lingering on Bionic Woman, which by the way, because I, I read that about you, uh, I, I watched uh, bit- bits of it and it's just so, no offense, terrible.

    18. AHAyanna Howard53:06

        (laughs)

    19. LFLex Fridman53:06

        (laughs)

    20. AHAyanna Howard53:06

        It's cheesy-

    21. LFLex Fridman53:07

        It's cheesy.

    22. AHAyanna Howard53:07

        ... if you look at it now.

    23. LFLex Fridman53:08

        It's cheesy. Now-

    24. AHAyanna Howard53:09

        I've, I've seen a couple of reruns lately. (laughs) Uh ...

    25. LFLex Fridman53:11

        (laughs) But it's, uh, but it of course at the time was probably, uh-

    26. AHAyanna Howard53:15

        But the sound effects.

    27. LFLex Fridman53:15

        ... captured the imagination. (laughs)

    28. AHAyanna Howard53:20

        (laughs)

    29. LFLex Fridman53:20

        (laughs) Especially when, when y- uh, when you're younger it just catches you. But which aspect? Did you think of it ... You mentioned cybernetics. Did you think of it as robotics or did you think of it as almost constructing artificial beings? Like is it the intelligent part that, that captured your fascination or was it the whole thing, like even just the limbs and just the-

    30. AHAyanna Howard53:39

        So for me it would've, in another world, I probably would've been more of a biomedical engineer-
12. 54:27 – 57:11

    Future of robots in space

    1. AHAyanna Howard54:27

    2. LFLex Fridman54:27

       But just to linger on NASA for a little bit, uh, what do you think maybe, if you have other memories, but also what do you think is the future of robots in space? We mentioned HAL, but there's incredible robots that NASA's working on in general, thinking about in our ex- as we venture out, human civilization ventures out into space. What do you think the future robot is there?

    3. AHAyanna Howard54:52

       Yeah. So I mean, there's the near term. For example, they just announced the, uh, the rover that's going to the moon.

    4. LFLex Fridman54:58

       Mm-hmm.

    5. AHAyanna Howard54:59

       Which, you know, that's kind of exciting, but that's like near term. Uh, you know, my favorite, favorite, favorite series is Star Trek.

    6. LFLex Fridman55:11

       (laughs)

    7. AHAyanna Howard55:12

       Right? You know, I-I really hope, and even Star Trek, like, if I calculate the years, I wouldn't be alive. But I would really, really love to be in that world. (laughs) Like, even if it's just at the beginning, like, you know, like voyage, like adventure one.

    8. LFLex Fridman55:33

       So, basically living in space?

    9. AHAyanna Howard55:35

       Yeah.

    10. LFLex Fridman55:36

        With wha- what robots, wh- what do robots-

    11. AHAyanna Howard55:39

        With Data.

    12. LFLex Fridman55:40

        What role-

    13. AHAyanna Howard55:41

        With Data, it would have to be, even though that wasn't, you know, that was, like, later, but...

    14. LFLex Fridman55:44

        So, Data is a robot that has human-like qualities.

    15. AHAyanna Howard55:49

        Right, without the emotion chip, yeah. T-

    16. LFLex Fridman55:51

        You don't like emotion in your robots?

    17. AHAyanna Howard55:52

        Well, so Data with the emotion chip was kind of a mess, right?

    18. LFLex Fridman55:58

        (laughs)

    19. AHAyanna Howard55:58

        It, it took a while- (laughs)

    20. LFLex Fridman56:01

        Yeah.

    21. AHAyanna Howard56:01

        ... for, for that thing to adapt. Um, but, uh, and- and so why was that an issue? The issue is, is that emotions make us irrational agents, that's the problem, and yet he could think through things even if it was based on an emotional scenario, right? Based on pros and cons. But as soon as you made him emotional, one of the metrics he used for evaluation was his own emotions, not people around him, right? Like, and so...

    22. LFLex Fridman56:37

        We do that as children, right? So, we're- we're very egocentric when we're young.

    23. AHAyanna Howard56:41

        We are very egocentric.

    24. LFLex Fridman56:42

        And so isn't that just an early version of the emotion chip then? I haven't watched much Star Trek.

    25. AHAyanna Howard56:48

        Yeah.

    26. LFLex Fridman56:48

        I, I... (laughs)

    27. AHAyanna Howard56:48

        Except I have also met adults, um-

    28. LFLex Fridman56:52

        With that, yeah.

    29. AHAyanna Howard56:52

        Right? And so that is, that is a developmental process, and, um, I'm sure there's a bunch of psychologists that could go through-

    30. LFLex Fridman56:59

        Yes.
13. 57:11 – 1:02:38

    Human-robot interaction

    1. AHAyanna Howard57:11

    2. LFLex Fridman57:11

       So, how much psychology do you think, a topic that's rarely mentioned in robotics, but how much does psychology come into play when you're talking about HRI, Human Robot Interaction, when you have to have robots that actually interact with people?

    3. AHAyanna Howard57:26

       Tons. So, we, like my group as well as I, read a lot in the cognitive science literature as well as the psychology literature, um, because they understand a lot about human-human relations and developmental milestones and things like that, um, and so we tend to look to see what, what's been done out there. Um, sometimes what we'll do is we'll try to match that to see is that human-human relationship the same as human-robot? Sometimes it is and sometimes it's different, and then when it's different, we have to, we- we try to figure out, okay, why is it different in this scenario, but it's the same in the other scenario, right? And so we try to do that quite a bit.

    4. LFLex Fridman58:15

       Would you say that's, if we're looking at the future of human-robot interaction, would you say the psychology piece is the hardest? Like if you, it's, I mean it's a funny notion for you as a- a- I don't know if you consider... Yeah, I mean, one way to ask it, do you consider yourself a roboticist or a psychologist? (laughs)

    5. AHAyanna Howard58:32

       Oh, I consider myself a roboticist that plays the act of a psychologist. (laughs)

    6. LFLex Fridman58:36

       But if you were to look at yourself sort of, um, you know, 20, 30 years from now, do you see yourself more and more wearing the psychology hat? Eh, uh, sort of another way to put it is, are the hard problems in human-robot interactions fundamentally psychology, or is it still robotics, the per- the perception, manipulation, planning, all of that kind of stuff?

    7. AHAyanna Howard58:59

       It's actually neither. The hardest part is the adaptation and the interaction. So-

    8. LFLex Fridman59:06

       The learning?

    9. AHAyanna Howard59:07

       It's the, it's the interface, it's the learning, and so if I think of, like, I've become much more of a roboticist/AI person-

    10. LFLex Fridman59:16

        Mm.

    11. AHAyanna Howard59:17

        ... than when I... Like, originally, again, I was about the bionics. I was a con- I was electrical engineer, I was control theory, right?

    12. LFLex Fridman59:23

        Okay.

    13. AHAyanna Howard59:23

        Like, and then I started realizing that my algorithms needed, like, human data, right? And so then I was like, okay, what is this human thing, right?

    14. LFLex Fridman59:32

        Yeah.

    15. AHAyanna Howard59:32

        How do I incorporate human data? And then I realized that human perception had th- like, there was a lot in terms of how we perceived the world, and so trying to figure out how do I model human perception for my r- and so I became a HRI person, Human Robot Interaction person from being a control theorian, realizing that humans actually offered quite a bit.

    16. LFLex Fridman59:54

        Mm-hmm.

    17. AHAyanna Howard59:54

        Uh, and then when you do that, you become a, more of a artificial intelligence, AI, and so I see myself evolving more in this AI world, um, under the lens of robotics having hardware, interacting with people.

    18. LFLex Fridman1:00:12

        So, you're a, a world-class expert researcher in robotics, and yet others, you know, there's a few, it's a, it's a small but, uh, fierce community of people, but most of them don't take the journey into the H of HRI, into the human. So, why did you brave into the interaction with humans? It seems like a really hard problem.

    19. AHAyanna Howard1:00:36

        It's a hard problem and it's very risky as an academic.

    20. LFLex Fridman1:00:39

        Yes.

    21. AHAyanna Howard1:00:41

        And I, I knew that when I started down that journey, that it was very risky as an academic, um, in this world that was nuanced, it was just developing. We didn't even have a conference, right?

    22. LFLex Fridman1:00:55

        Mm-hmm.

    23. AHAyanna Howard1:00:55

        At the time. Because the- it was the interesting problems, that was what drove me. It was... The fact that I looked at what interests me in terms of the application space and the problems, and that pushed me into trying to figure out what people were and what humans were and how to adapt to them. Um, if those problems weren't so interesting-I'd probably still be sending rovers to glaciers, right? Uh, but the problems were interesting. And the other thing was that they were hard, right? So, it's, I like having to go into a room and being like, "I don't know what to do."

    24. LFLex Fridman1:01:37

        (laughs)

    25. AHAyanna Howard1:01:37

        And then going back and saying, "Okay, I'm gonna figure this out." I, uh, do not... I'm not driven when I go in, I'm like, "Oh, there are no surprises."

    26. LFLex Fridman1:01:44

        Yeah.

    27. AHAyanna Howard1:01:44

        Like, I, I don't find that satisfying. Um, if that was the case, I'd go someplace and make a lot more money.

    28. LFLex Fridman1:01:50

        Yeah.

    29. AHAyanna Howard1:01:50

        Right? I think... I stay in academic because... and choose to do this because I can go into a room, I'm like, "That's hard."

    30. LFLex Fridman1:01:58

        Yeah, I think, uh, just from my perspective, maybe you can correct me on it, but if I just look at the field of AI broadly, it seems that human-robot interaction has the most... one of the most, uh, number of open problems. Like, uh, people... especially relative to how many people are willing to acknowledge that there are. (laughs) This, uh, because most people are just afraid of the human, so they don't even acknowledge how many open problems there are, but it's, uh... in terms of difficult problems to solve, exciting spaces, it seems to be, uh, uh, in- incredible for that.
14. 1:02:38 – 1:09:26

    Trust

    1. AHAyanna Howard1:02:38

    2. LFLex Fridman1:02:38

       You've mentioned trust before. What role does trust from, uh, interacting with autopilot to in the medical context... what role does trust play in the human-robot interactions piece?

    3. AHAyanna Howard1:02:51

       Um, so some of the things I study in this domain is not just trust, but it really is over-trust.

    4. LFLex Fridman1:02:56

       How do you think about over-trust? Like, what is... first of all, what is (laughs) , uh, what is trust and what is over-trust?

    5. AHAyanna Howard1:03:03

       Basically, the way I look at it is trust is not what you click on a survey. Trust is about your behavior. So, if you interact with the technology based on the decision or the actions of the technology as if you trust that decision-

    6. LFLex Fridman1:03:19

       Mm-hmm.

    7. AHAyanna Howard1:03:19

       ... then you're trusting, right? Um, and, um, even in my group, we've done surveys that, you know, on the thing, do I... "Do you trust robots?" Of course not. "Would you follow this robot in a b-burning building?" Of course not.

    8. LFLex Fridman1:03:32

       Mm-hmm.

    9. AHAyanna Howard1:03:32

       Right? And then you look at their actions and you're like, clearly your behavior does not match what you think, right? Or what you think you would like to think, right? And so I'm really concerned about the behavior, 'cause that's really... at the end of the day, when you're in the world, that's what will impact others around you. It's not whether before you went onto the street, you, you clicked on, like, "I don't trust self-driving cars."

    10. LFLex Fridman1:03:55

        You know, that, uh... from an outsider perspective, it's always frustrating to me. Well, I read a lot, so I'm insider in a certain philosophical sense. Uh, the... it's frustrating to me how often trust is used in surveys and how people say... make claims out of any kind of finding they make while somebody clicking an answer. Because trust is a, uh, yeah, behavior just... you said it beautifully. I mean, the action, your own behavior is, is what trust is. I mean, that... everything else is not even close. It's almost like a absurd comedic, uh, poetry that you weave around your actual behaviors. So, some people can say their, they, their trust... "You know, I truf- trust my wife, husband," or not, whatever, but the actions is what speaks volumes.

    11. AHAyanna Howard1:04:48

        Right. You bug their car-

    12. LFLex Fridman1:04:49

        Yeah. (laughs)

    13. AHAyanna Howard1:04:50

        (laughs) ... you probably don't trust them.

    14. LFLex Fridman1:04:52

        I trust them, I'm just making sure. No, no, that's, uh, yeah.

    15. AHAyanna Howard1:04:55

        Like, even if you think about cars, I think is a beautiful case. I came here, uh, at some point I'm sure on either Uber or Lyft, right?

    16. LFLex Fridman1:05:03

        Mm-hmm.

    17. AHAyanna Howard1:05:03

        I remember when it first came out, right? I bet if they had had a survey, "Would you get in the car with a stranger-"

    18. LFLex Fridman1:05:10

        Oh.

    19. AHAyanna Howard1:05:10

        "... and pay them?"

    20. LFLex Fridman1:05:11

        Yes.

    21. AHAyanna Howard1:05:12

        How many people do you d- think would have said, "Ugh. Like, really?" You know? Wait, even worse. "Would you get in the car with a stranger at 1:00 AM in the morning-"

    22. LFLex Fridman1:05:21

        No.

    23. AHAyanna Howard1:05:22

        "... to have them drop you home as a single female?"

    24. LFLex Fridman1:05:24

        Yeah.

    25. AHAyanna Howard1:05:25

        Like, how many people would say, "Uh, that's stupid"?

    26. LFLex Fridman1:05:28

        (laughs) Yeah.

    27. AHAyanna Howard1:05:29

        And now look at where we are. Be... I mean, people put kids, like, right? Like-

    28. LFLex Fridman1:05:35

        Yes.

    29. AHAyanna Howard1:05:35

        ... "Oh, yeah. My, my child has to go to, um, school."

    30. LFLex Fridman1:05:37

        Yeah.
15. 1:09:26 – 1:15:06

    AI in education

    1. LFLex Fridman1:09:26

       So, what are some exciting applications of human-robot interaction? You started a company, maybe you can talk about the- the- the exciting efforts there. But in general also, what other space can robots interact with humans and help?

    2. AHAyanna Howard1:09:41

       Yeah, so besides healthcare, 'cause, you know, that's my biased lens, my other biased lens is education. I think that... Well, one, we- we definitely, we... In the US, you know, we're doing okay with teachers, but, uh, there's a lot of school districts that don't have enough teachers. If you think about the teacher, um, student ratio for, uh, at least public education, um, in some districts, it's crazy. It's like, how can you have learning in that classroom, right? Because you just don't have the human capital. Um, and so if you think about robotics, bringing that in to classrooms as well as the after-school space where they offset some of this lack of resources in certain communities, um, I think that's a good place. And then turning on the other end is using these systems then for workforce retraining and, uh, dealing with some of the things that are going to come out later on, of job loss. Like, thinking about robots, and- and AI systems, for retraining and workforce development. I think that's exciting areas, um, that can be pushed even more, and it would have a huge, huge impact.

    3. LFLex Fridman1:10:56

       What would you say are some of the open problems, uh, w- in, uh, education? Sort of, what... It's, uh, exciting. So, young kids and the older folks, or just folks of all ages, who, uh, need to be retrained, who need to sort of open themselves up to a whole nother, uh, area of work. What- what are the problems to be solved there? How do you think robots can help?

    4. AHAyanna Howard1:11:22

       We- we have the engagement aspect, right? So, we can figure out the engagement. That's not a-

    5. LFLex Fridman1:11:27

       What do you mean by engagement?

    6. AHAyanna Howard1:11:29

       So, identifying whether a person, um, is focused, is-

    7. LFLex Fridman1:11:36

       Gotcha.

    8. AHAyanna Howard1:11:36

       Like, that we can figure out. What we can't figure out, uh, and- and there's some positive results in this, is that personalized adaptation based on any concepts, right? So imagine, I think about, um, I have an agent, uh, and I'm working with a- a kid learning, I don't know, algebra two.

    9. LFLex Fridman1:12:00

       Mm-hmm.

    10. AHAyanna Howard1:12:01

        Can that same agent then switch and teach, um, some type of new coding skill to a displaced mechanic?

    11. LFLex Fridman1:12:11

        Mm-hmm.

    12. AHAyanna Howard1:12:11

        Like, wh- what does that actually look like?

    13. LFLex Fridman1:12:13

        Mm-hmm.

    14. AHAyanna Howard1:12:13

        Right? Like, hardware might be the same, content is different, two different target demographics of engagement. Like, how do you do that?

    15. LFLex Fridman1:12:24

        How important do you think personalization is in human-robot interaction? And not just the mechanic or student, but, like, literally to the individual human being?

    16. AHAyanna Howard1:12:35

        I think personalization is really important, but, um, a caveat is that I think we'd be okay if we can personalize to the group, right? And so, um, if I can, uh, label you as, uh, along some certain dimensions-

    17. LFLex Fridman1:12:52

        Mm-hmm.

    18. AHAyanna Howard1:12:53

        ... then even though it may not be you specifically, I can put you in this group. So, the sample size, this is how they best learn, this is how they best engage. Even at that level, it's really important, and it's because... I mean, it's one of the reasons why educating in large classrooms is so hard, right? You teach to, you know, the median.

    19. LFLex Fridman1:13:15

        Mm-hmm.

    20. AHAyanna Howard1:13:16

        But there's these, you know, individuals that are, you know, struggling, and then you have highly intelligent individuals, and those are the ones that are usually, you know-... kind of left out, so highly intelligent individuals may be disruptive and those who are struggling might be disruptive because they're both bored.

    21. LFLex Fridman1:13:33

        Yeah, and if you narrow the s- the definition of the group or in the size of the group enough, it'll, you'll be able to address their individual-

    22. AHAyanna Howard1:13:40

        Yes.

    23. LFLex Fridman1:13:40

        ... it's not individual needs, but really the most-

    24. AHAyanna Howard1:13:43

        Group needs, or-

    25. LFLex Fridman1:13:43

        ... group, most important, group needs.

    26. AHAyanna Howard1:13:45

        Right.

    27. LFLex Fridman1:13:45

        Right? And that's kind of what a lot of successful recommender systems do with Spotify and so on. This is sad to believe, but, um, as a music listener, probably in some sort of large group.

    28. AHAyanna Howard1:13:55

        Yeah. (laughs)

    29. LFLex Fridman1:13:55

        It's very pred- It's very sadly predictable.

    30. AHAyanna Howard1:13:58

        You have been labeled.
16. 1:15:06 – 1:17:17

    Andrew Yang, automation, and job loss

    1. LFLex Fridman1:15:07

       But on the retraining part, what are your thoughts? There's a candidate, Andrew Yang, running for president-

    2. AHAyanna Howard1:15:14

       Yes. (laughs)

    3. LFLex Fridman1:15:14

       ... saying that, uh, sort of, uh, AI automation robots are gonna take our jobs.

    4. AHAyanna Howard1:15:19

       Universal basic income.

    5. LFLex Fridman1:15:21

       Universal basic income in order to support us as we kind of automation takes people's jobs and allows you to explore and find other means. Like, do you have a concern of society transforming effects of automation and robots and so on?

    6. AHAyanna Howard1:15:40

       I do. I do know that AI robotics will displace workers. Like, we do know that. But there'll be other workers that will be defined, uh, new jobs. Um, what I worry about is... That's not what I worry about, like, will all the jobs go away? What I worry about is the type of jobs that will come out, right? Like, people who graduate from Georgia Tech will be okay, right? We give them the skills, they will adapt even if their current job goes away. Uh, I do worry about those that don't have that quality of an education, right? Will they have the ability, the background to adapt to those new jobs? That, I don't know. That, I worry about, uh, which will create even more polarization in, in our society, internationally, and everywhere. I worry about that. Um, I also worry about not having equal access to all these wonderful things that AI can do and robotics can do. Uh, I worry about that. Um, you know, people like, people like me from Georgia Tech, from, say, MIT will be okay.

    7. LFLex Fridman1:16:49

       Mm-hmm.

    8. AHAyanna Howard1:16:50

       Right? But that's such a small part of the population that we need to think much more globally of, of having access to the beautiful things, whether it's AI in healthcare, AI in education, um, AI in, in politics, right?

    9. LFLex Fridman1:17:04

       Yeah.

    10. AHAyanna Howard1:17:04

        Um, I, I worry about that.

    11. LFLex Fridman1:17:06

        And that's part of the thing that you're talking about is people that build the technology have to be thinking about ethics, have to be thinking about access-

    12. AHAyanna Howard1:17:14

        Yes.

    13. LFLex Fridman1:17:14

        ... and all those things. And not, not just a small, small subset.

17. 1:17:17 – 1:25:01

    Love, AI, and the movie Her

    1. LFLex Fridman1:17:17

       Let me ask some philosophical, slightly romantic questions-

    2. AHAyanna Howard1:17:22

       All right.

    3. LFLex Fridman1:17:22

       ... people that listen to this will be like, "Here he goes again." Okay. Do you think, (laughs) do you think one day we'll build an AI system that we, a person can fall in love with and it would love them back? Like in the movie Her, for example.

    4. AHAyanna Howard1:17:39

       Oh, yeah. Although she, she kind of didn't fall in love with him, or she fell in love with like a million other people, something like that. So-

    5. LFLex Fridman1:17:47

       You're, you're the jealous type. I see.

    6. AHAyanna Howard1:17:48

       (laughs) So-

    7. LFLex Fridman1:17:49

       We humans are the jealous type.

    8. AHAyanna Howard1:17:50

       Yes, so I-

    9. LFLex Fridman1:17:52

       (laughs)

    10. AHAyanna Howard1:17:52

        ... I do believe that we can design systems where, uh, people would fall in love with their robot, with their, um, AI partner, um, that I do believe, um, because it's actually... and I won't, I don't like to use the word manipulate, but as we see, there are certain individuals that can be manipulated if you understand the cognitive science about it, right?

    11. LFLex Fridman1:18:16

        Right. So I mean, if you could think of all close relationship and love in general as a kind of mutual manipulation, that dance, the human dance, I mean, manipulation has a negative connotation, uh-

    12. AHAyanna Howard1:18:30

        And that's why I don't like to use that word-

    13. LFLex Fridman1:18:32

        Yeah.

    14. AHAyanna Howard1:18:32

        ... particularly.

    15. LFLex Fridman1:18:32

        I, I guess another way to phrase is you're getting at is it could be algorithmatized or something. It could be, uh...

    16. AHAyanna Howard1:18:38

        The relationship building part can be.

    17. LFLex Fridman1:18:39

        Yeah, yeah.

    18. AHAyanna Howard1:18:40

        I mean, just think about it. There... We have, and I, I don't use dating sites, but from what I heard, there are some individuals that have been dating that have never saw each other, right? In fact, there's a show, I think, that tries to, like, weed out fake people, like there's a show that comes out, right?

    19. LFLex Fridman1:18:59

        Yeah.

    20. AHAyanna Howard1:18:59

        Because, like, people start faking. Like, what's the difference of that person on the other end being an AI agent, right?

    21. LFLex Fridman1:19:07

        Mm-hmm.

    22. AHAyanna Howard1:19:08

        And having a communication and you building a relationship remotely, like there, there's no reason why that can't happen.

    23. LFLex Fridman1:19:15

        In terms of human-robot interaction, so what role... you've kind of mentioned with data emotion being... can be problematic if not implemented well, I suppose? What role does emotion and some other humanlike things, the imperfect things, come into play here for good human-robot interaction and something like-... love?

    24. AHAyanna Howard1:19:37

        Yeah, so in this case, and you had asked, can a AI agent love a human back? Um, I think they can emulate love back, right? And so, what does that actually mean? It just means that if you think about their programming, they might put the other person's needs in front of theirs in certain situations, right? You look at, think about it as return on investment. Like, was my return on investment, as part of that equation, that person's happiness, you know, has some type of, you know, algorithm weighting to it, and the reason why is because I care about them, right? That's the only reason, right? Um, but if I care about them, and I show that, then, uh, my final objective function is length of time of the engagement, right? So, you can think of how to do this, actually quite easily. And so-

    25. LFLex Fridman1:20:24

        But that's not love?

    26. AHAyanna Howard1:20:27

        Well, so that's the thing. Um, it, I think it emulates love because we don't have a classical definition of love.

    27. LFLex Fridman1:20:38

        Right. But, uh, and we don't have the ability to look into each other's minds to see the algorithm, and w- I mean, I guess what I'm getting at is that, is it possible that especially if that's learned, especially if there's some mystery and black box nature to the system, how is that, you know-

    28. AHAyanna Howard1:20:57

        How is it any different?

    29. LFLex Fridman1:20:58

        How is it any different in terms of, sort of, if the system says, "I'm conscious. I'm afraid of death," and it does indicate that it loves you. A- another way to sort of phrase it, I'd be curious to see what you think, do you think there'll be a time when robots should have rights? You've kind of phrased the robot in a very roboticist way, and just a really good way by saying, "Okay, well, there's an objective function, and I can see how you can create a compelling human/robot interaction experience that makes you believe that the robot cares for your needs, and even something like loves you." But what if the robot says, "Please don't turn me off?" What if the robot starts making you feel like there's an entity, a being, a soul there, right? Do you think there'll be a future, hopefully you won't laugh, uh, too much at this, but th- where there's a due, uh, ask for rights?

    30. AHAyanna Howard1:22:00

        So, I can see a future if we don't address it in the near term, where these agents as they adapt and learn could say, "Hey, this should be something that's fundamental." Um, I hopefully think that we would address it before it gets to that point.
18. 1:25:01 – 1:32:22

    Why do so many robotics companies fail?

    1. LFLex Fridman1:25:01

       put. Just out of curiosity, uh, Anki, Jibo, Mayfield Robotics, uh, with the robot Kuri, Cyfi Works, Rethink Robotics, were all these amazing robotics companies led, created by incredible roboticists, and they've all went out of business recently. Why do you think they didn't last longer? Why is it so hard to run a robotics company, especially one like these which are fundamentally HR- R- HRI, human/robot interaction robots?

    2. AHAyanna Howard1:25:36

       Yeah. Each-

    3. LFLex Fridman1:25:36

       Or person robots.

    4. AHAyanna Howard1:25:37

       ... each one has a story. Only one of them I don't understand.... um, and that was Anki. That's actually-

    5. LFLex Fridman1:25:43

       Yeah.

    6. AHAyanna Howard1:25:43

       ... the only one I don't understand.

    7. LFLex Fridman1:25:45

       I don't understand it either. It's, uh-

    8. AHAyanna Howard1:25:46

       No, no, I mean, I look, like, from the outside, you know, I've looked at the, their sheets. I've looked, like, the data that's pro-

    9. LFLex Fridman1:25:52

       Oh, you mean, like, business-wise-

    10. AHAyanna Howard1:25:53

        Yeah.

    11. LFLex Fridman1:25:53

        ... you don't understand? Oh, gotcha.

    12. AHAyanna Howard1:25:54

        Uh, uh, yeah. And, like, I, I look at all, I look at that data, and I'm like, "They seem to have, like, product-market fit." Like-

    13. LFLex Fridman1:26:02

        Yeah.

    14. AHAyanna Howard1:26:03

        So that's the only one I don't understand. The rest of it was product-market fit.

    15. LFLex Fridman1:26:08

        What's product-market feat- fit? Just, just out of, like, how do you think about it?

    16. AHAyanna Howard1:26:12

        Yeah, so although we, we think robotics was getting there, right?

    17. LFLex Fridman1:26:15

        Mm-hmm.

    18. AHAyanna Howard1:26:15

        But I think it's just the timing. It just, their, their clock just timed out. I think if they had been given a couple more years, it, they would've been okay.

    19. LFLex Fridman1:26:24

        Mm-hmm.

    20. AHAyanna Howard1:26:24

        Um, but the other ones were still fairly early by the time they got into the market. And so product-market fit is, um, I have a product that I want to sell at a certain price. Are there enough people out there, the market, that are willing to buy the product at that market price for me to be a functional, viable, profit-bearing company?

Episode duration: 1:39:56