Lex Fridman PodcastRosalind Picard: Affective Computing, Emotion, Privacy, and Health | Lex Fridman Podcast #24
EVERY SPOKEN WORD
105 min read · 20,961 words- 0:00 – 1:55
What affective computing really includes (beyond emotion recognition)
- LFLex Fridman
The following is a conversation with Rosalind Picard. She's a professor at MIT, director of the Affective Computing Research Group at the MIT Media Lab, and co-founder of two companies, Affectiva and Empatica. Over two decades ago, she launched the field of affective computing with her book of the same name. This book described the importance of emotion in artificial and natural intelligence, the vital role emotional communication has to the relationship between people in general, and human-robot interaction. I really enjoyed talking with Roz over so many topics, including emotion, ethics, privacy, wearable computing, and her recent research in epilepsy, and even love and meaning. This conversation is part of the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D. And now, here's my conversation with Rosalind Picard. More than 20 years ago, you've coined the term affective computing and led a lot of research in this area since then. As I understand, the goal is to make the machine detect and interpret the emotional state of a human being, and adapt the behavior of the machine based on the emotional state. So, how has your understanding of the problem space defined by affective computing changed in the past 24 years? So it's the scope, the applications, the challenges, uh, what's involved. How has that evolved over the years?
- RPRosalind Picard
Yeah. Actually, originally when I defined the term affective computing, it was a bit broader than just recognizing and responding intelligently to human emotion, although those are probably the two pieces that we've worked on the hardest. The original concept also encompassed machines that would have mechanisms that functioned like human emotion does inside them.
- LFLex Fridman
Ah.
- 1:55 – 5:02
Clippy as a lesson in emotionally unintelligent design
- RPRosalind Picard
It would be any computing that relates to, arises from, or deliberately influences human emotion. So, the human-computer interaction part is the part that people tend to see. Like, if I'm, you know, really ticked off at my computer and I'm scowling at it and I'm cursing at it and it just keeps acting smiling and happy like that little paper clip used to do...
- LFLex Fridman
Yeah.
- RPRosalind Picard
... like dancing, winking, um, that kind of thing just makes you even more frustrated, right? And I thought, "That stupid thing needs to see my affect. And if it's gonna be intelligent," which Microsoft researchers had worked really hard on, it actually had some of the most sophisticated AI in it at the time, "If that thing's gonna actually be smart, it needs to respond to me and you."
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
"And we can, um, send it very different signals."
- LFLex Fridman
So by the way, just a quick interruption. The Clippy, maybe it's in Word, uh, 95, 98...
- RPRosalind Picard
Mm-hmm.
- LFLex Fridman
I don't remember when it was born. But, uh, m- many people... Do you find yourself with that reference, that people recognize what you're talking about still to this point?
- RPRosalind Picard
The... I don't expect the newest students to these days, but I've mentioned it to a lot of audiences, like, "How many of you know this Clippy thing?" And still the majority of people seem to know it.
- LFLex Fridman
So Clippy kind of looks at maybe natural language processing, what you were typing, and tries to help you complete, I think. I, I don't even remember what Clippy wa- was except annoying. (laughs)
- RPRosalind Picard
Yeah. It, it tried... Some people actually liked it.
- LFLex Fridman
Well, I miss it.
- RPRosalind Picard
I would hear those stories. You miss it?
- LFLex Fridman
Well, I miss the... the, the annoyance. It felt like, uh, there's a, there's an element-
- RPRosalind Picard
Someone was there.
- LFLex Fridman
Somebody was there, and we were in it together.
- RPRosalind Picard
Yeah.
- LFLex Fridman
And they were annoying.
- RPRosalind Picard
Yeah.
- LFLex Fridman
It's like a, it's like a puppy that just doesn't get it.
- RPRosalind Picard
Right.
- LFLex Fridman
That keeps ripping up the couch kind of thing.
- RPRosalind Picard
And in fact, they could have done it smarter like a puppy. If they had done, like, if when you yell- yelled at it or cursed at it, if it had put its little ears back and its tail down and shirked off, probably people would have wanted it back, right? But instead, when you yelled at it, what did it do? It smiled, it winked, it danced, right?
- LFLex Fridman
Yeah.
- RPRosalind Picard
If somebody comes to my office and I yell at them and they started smiling, winking, and dancing, I'm like, "I never want to see you again." So Bill Gates got a standing ovation when, uh, he said it was going away-
- LFLex Fridman
(laughs)
- RPRosalind Picard
... 'cause people were so ticked. It was so emotionally unintelligent, right? It was intelligent about whether you were writing a letter, what kind of help you needed for that context. It was completely unintelligent about, "Hey, if you're annoying your customer, don't smile in their face when you do it." So that kind of mismatch was something that developers just didn't think about. And intelligence at the time was really all about, uh, math and language and chess and, you know, games, uh, problems that could be pretty well-defined. Social-emotional interaction is much more complex than chess or Go or any of the games I, that people are trying to solve. And in order to understand that required skills that most people in computer science actually were lacking personally.
- 5:02 – 5:54
Have computer scientists gotten more empathetic? Diversity and the human factor
- LFLex Fridman
Well, let's talk about computer science, if things have gotten better since, since the work, since the message, uh, since you've really launched a field with a lot of research work in this space. I still find, as a person like yourself who's deeply passionate about human beings and, and yet am in computer science, there still seems to be a lack of... sorry to say, empathy for... (laughs)
- RPRosalind Picard
(laughs) Um-
- LFLex Fridman
In, uh, in us computer scientists.
- RPRosalind Picard
Yeah. Well-
- LFLex Fridman
Or has it gotten better?
- RPRosalind Picard
Let, let's just say there's a lot more variety among computer scientists these days. Computer scientists are a much more diverse group today than they were 25 years ago. And that's good. Uh, we need all kinds of people to become computer scientists so that computer science reflects more what society needs. And, you know, there's brilliance among every personality type, so it need not be, uh, limited to people who prefer computers to other people.
- 5:54 – 7:52
How hard is emotional intelligence for machines, really? Limits of narrow context
- LFLex Fridman
How hard do you think it is? How... Your view of how difficult it is to recognize emotion or to create a deeply, uh, emotionally intelligent interaction, has it gotten easier or harder as you've explored it further?
- RPRosalind Picard
Yeah.
- LFLex Fridman
And how far away are we from cracking this...... the, if, if you think of Turing test solving the intelligence, looking at the Turing test for emotional intelligence?
- RPRosalind Picard
I think it is as difficult as I thought it was gonna be. I think my prediction of its difficulty is spot on. I think the time estimates are always hard because they're always a function of society's, uh, love and hate of a particular topic.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
If society gets excited and you get, you know, hundreds of, uh, and you get thousands of researchers working on it for a certain application, that application gets solved really quickly. The general intelligence, the lack, the c- the computer's complete lack of ability to have, uh, awareness of what it's doing, the fact that it's not conscious, the fact that there's no signs of it becoming conscious, the fact that it doesn't read between the lines, those kinds of things that we have to teach it explicitly what other people pick up implicitly, we don't see that changing yet. There aren't breakthroughs yet that lead us to believe that that's gonna go any faster, which means that it's still gonna be kinda stuck with a lot of limitations, where it's probably only gonna do the right thing in very limited, narrow, pre-specified contexts, where we can prescript- prescribe pretty much what's, what's gonna happen there. So I don't see the, uh, you know, it's hard to predict a date, because when people don't work on it, it's infinite.
- LFLex Fridman
(laughs)
- 7:52 – 11:08
Putting on the brakes: surveillance misuse and the China scenario
- RPRosalind Picard
When (laughs) everybody works on it, you get a nice piece of it, uh, you know, well solved in a short amount of time. I actually think there's a more important issue right now than the difficulty of it, and that's causing some of us to put the brakes on a little bit. Usually we're all just like, "Step on the gas. Let's go faster." Uh, this is causing us to pull back and put the brakes on, and that's, uh, the way that some of this technology is being used in places like China right now.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
And that worries me so deeply that it's causing me to pull back myself on a lot of the things that we could be doing, and, uh, try to get the community to think a little bit more about, okay, if we're gonna go forward with that, how can we do it in a way that puts in place safeguards that protects people?
- LFLex Fridman
So the technology we're re- referring to is just when a computer senses the human being, like the human face?
- RPRosalind Picard
Yeah.
- LFLex Fridman
Right?
- RPRosalind Picard
Yeah.
- LFLex Fridman
And so what... There's a lot of exciting things there, like forming-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... a deep connection with a human being. But so what are your worries how that could go wrong? Uh, is it in terms of privacy? Is it in terms of, uh, uh, o- o- other kinds of-
- RPRosalind Picard
Let, let's dig into privacy.
- LFLex Fridman
... more subtle things? Okay.
- RPRosalind Picard
So here in the US, if I'm watching a video of, say, a political leader, and, and in the US, we're quite free, as we all know, to even criticize the, you know, the President of the United States, right?
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Here, that's not a shocking thing. It happens, uh, you know, about every five seconds, right? (laughs)
- LFLex Fridman
(laughs)
- RPRosalind Picard
But in, um, China, uh, what happens if you criticize the leader of the government, right? And so people are very careful not to, to do that. However, what happens if you're simply watching a video and you make a facial expression that's- shows a little bit of skepticism?
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Right? Well, in, you know, here, we're completely free to do that. In fact, we're free to fly off the handle, uh, and say anything we want, uh, usually. I mean, there are some restrictions, you know?
- LFLex Fridman
Yeah.
- RPRosalind Picard
When, when the athlete does this, um, as part of the national broadcast, maybe the teams get a little, uh, unhappy about picking that forum to do it, right? But that, that's more a question of judgment. Um, we, we have these, uh, freedoms, and in places that don't have those freedoms, what if our technology can read your underlying affective state? What if our technology can read it even non-contact? What if our technology can read it without your, uh, prior consent?
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
And here in the US, um, my first company we started, Affectiva-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... we have worked super hard to turn away money and opportunities that try to read people's affect without their prior informed consent. And even the software that is licensable, you have to sign things saying you will only use it in certain ways, which essentially is get people's buy-in, right?
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Don't, don't do this without people, uh, agreeing to it. There are other countries where they're not interested in people's buy-in. They're just gonna use it. They're gonna inflict it on you. And if you don't like it, you better not scowl in the direction of any censors.
- 11:08 – 12:36
Deepfakes, physiological signal extraction, and the need to ‘jam’ sensing
- LFLex Fridman
So one, let me just comment on a, on a small tangent. Do you know, with the, the idea of, uh, adversarial examples and deep fakes and so on... (laughs)
- RPRosalind Picard
Yeah.
- LFLex Fridman
What you bring up is actually the, the, in, uh, one sense, deep fakes provide a comforting protection that, uh, that y- you can no longer really trust that, uh, the video of your face was legitimate, and therefore you always have an escape clause if a government is trying... If a, uh, stable, balanced, uh, ethical government is trying to accuse you of something, at least you have protection. You can say it was fake news. This is a popular term now. Uh-
- RPRosalind Picard
Yeah, that's the general thinking of it. We, we know how to go into the video and see, for example, your heart rate and respiration, uh, and whether or not they've been tampered with, and we also can put like fake heart rate and respiration in your video-
- LFLex Fridman
Oh, interesting.
- RPRosalind Picard
... now, too. We, we decided we needed to do that, uh, after we-
- LFLex Fridman
(laughs)
- RPRosalind Picard
... after we developed a way to, to extract it, we decided we also needed a way to, to jam it.Right? And so the fact that we took time to do that other step too, right, that was time that I wasn't spending making the machine more affectively intelligent. And there's a choice in how we spend our time, which is now being swayed a little bit less by this goal and a little bit more, like, by concern about what's happening in society-
- LFLex Fridman
Mm-hmm.
- 12:36 – 22:22
Who benefits from AI? Inequality, incentives, and targeted regulation
- RPRosalind Picard
... and what kind of future do we want to build. And as we step back and say, "Okay, we don't just build AI to build AI to make Elon Musk more money or to make Amazon and Jeff Bezos more money."
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Good gosh, you know. That's, that's the wrong ethic. Uh, why are we building it? What, what is the point of building AI? It used to be it was driven by researchers in academia to get papers published (laughs) and to make a career for themselves and to do something cool, right?
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Like, 'cause maybe it could be done. Uh, now we realize that this is enabling rich people to get vastly richer.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
The poor are... The divide is even larger. And is that the kind of future that we want? Maybe we wanna think about... Maybe we wanna rethink AI. Maybe we wanna rethink the problems in society that are, um, causing the greatest inequity and rethink how to build AI that's not about a general intelligence, but that's about extending the intelligence and capability to the have-nots so that we close these gaps in society.
- LFLex Fridman
Do you hope that kind of stepping on the brake happens organically? Because I think still majority of the force behind AI is the desire to publish papers, is to make money-
- RPRosalind Picard
Right.
- LFLex Fridman
... without thinking about the why. Do, do you hope it happens organically? Is there room for regulation? Is, uh...
- RPRosalind Picard
Yeah, yeah, yeah. Great questions. I prefer the... You know, they talk about the carrot versus the stick. I definitely prefer the carrot to the stick. And, you know, in, in our free world, we... There's only so much stick, right? You're, (laughs) you're gonna find a way around it. I generally think less regulation is better. That said, even though my position is classically carrot, no stick, no regulation, I think we do need some regulations in this space. I do think we need regulations around pe- protecting people with their data, that you own your data, not Amazon, not Google. I would like to see people own their own data. I would also like to see the regulations that we have right now around lie detection being extended to emotion recognition in general. That right now you can't use a lie detector on an employee when you're... On, on a candidate when you're interviewing them for a job.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
I think similarly, we need to put in place protection around reading people's emotions without their consent and in certain cases, like characterizing them for a job and other opportunities. So, I'm also... I also think that when we're reading emotion that's predictive around mental health, that that should, even though it's not medical data, that that should get the kinds of protections that our medical data gets. Um, what most people don't know yet is right now, with your smartphone use and if you're wearing a sensor and you wanna learn about your stress and your sleep and your physical activity and how much you're using your phone and your social interaction, all of that non-medical data, when we put it together with machine learning, now called AI even though the founders of AI wouldn't have called it that, uh, that capability cannot only tell that you're calm right now or that you're getting a little stressed, uh, but it can also predict how you're likely to be tomorrow, if you're likely to be sick or healthy, happy or sad, stressed or calm.
- LFLex Fridman
Especially when you're tracking data over time.
- RPRosalind Picard
Especially when we're tracking a week of d- your data or more.
- LFLex Fridman
Do you have an optimism towards... You know, there are a lot of people on our, on our phones are worried about this camera that's looking at us. For the most part, on balance, do you, are you optimistic about the benefits that can be brought from that camera that's looking at billions of us, or-
- RPRosalind Picard
Uh...
- LFLex Fridman
... should we be more worried?
- RPRosalind Picard
I think we should be a little bit more worried about who's looking at us and listening to us. The device sitting on your countertop in your kitchen, whether it's, you know, Alexa or Google Home or Apple Siri, uh, these devices want to listen. While they say ostensibly to help us, and I think there are great people in these companies who do wanna help people. I... L- let me not brand them all bad. Uh, I'm a user of products from all, all of these companies I'm naming, all the A companies. Alphabet, (laughs) Apple-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... Amazon. They are awfully big companies, right? They have incredible power. And, you know, what if, what if China were to buy them, right, and suddenly all of that data were not part of free America, but all of that data were part of somebody who just wants to take over the world and you submit to them? And guess what happens if you so much as smirk the wrong way-
- LFLex Fridman
Mm.
- RPRosalind Picard
... when they say something that you don't like. Well, they have reeducation camps, right?
- LFLex Fridman
Right.
- RPRosalind Picard
That's a nice word for them. By the way, they have a surplus of organs for people who have surgery these days. They don't have an organ donation problem, 'cause they take your blood and they know you're a match. And the doctors are on record of, um, taking organs from people who are perfectly healthy and not, uh, prisoners. They're just simply not the favored ones of the government. Uh, and, you know, that's a pretty freaky, evil society. And we can use the word evil there.
- LFLex Fridman
I was born in the Soviet Union. I can certainly connect to the, uh, to the worry that you're expressing. At the same time, probably both you and I, and you very much so...You know, there's an exciting possibility that you can have a deep connection with a machine. (laughs)
- RPRosalind Picard
Yeah, yeah.
- LFLex Fridman
Right? So, uh...
- RPRosalind Picard
(laughs)
- LFLex Fridman
(laughs)
- 22:22 – 25:38
Should assistants read emotion? Suicide risk vs emotional manipulation for profit
- LFLex Fridman
So there's attention here. So when you expressed a really serious concern about privacy, about how governments can misuse the information, and there's the possibility of this connection. So let's, let's look at Alexa.
- RPRosalind Picard
Yeah. Yeah.
- LFLex Fridman
So a personal assistant's, uh, for the most part, as far as I'm aware, they ignore your emotion. They ignore even the context or the existence of you, the intricate, beautiful, complex aspects of who you are. Uh, except maybe aspects of your voice that help it recognize for, for speech recognition. Do you think they should move towards trying to understand your emotion?
- RPRosalind Picard
All of these companies are very interested in understanding human emotion. They, they want... More people are telling Siri every day they wanna kill themselves. They, Apple wants to know the difference between if a person is really suicidal versus if a person is just kind of fooling around with Siri, right? The words may be the same. The tone of voice and what surrounds those words is, is pivotal to understand if they should respond in a very serious way, bring help to that person, or if they should kind of jokingly tease back, you know, "Ah, you just wanna," you know, "sell me for something else," right? Like, like, how do you respond when somebody says that? Well, you know, you do wanna err on the side of being careful and taking it seriously. Uh, people wanna know if the person is happy or stressed, in part bec- well, so let me give you an altruistic reason and a, a business profit-motivated reason. And there are people in companies that operate on both principles. The altruistic people really care about their customers and really care about helping you feel a little better at the end of the day, and it would just make those people happy if they knew that they made your life better, if you came home stressed, and after talking with their product, you felt better. There are other people who maybe have studied the way affect affects decision-making and prices people pay, and they know...... you. I don't know if I should tell you, like the work of Jen Lerner at-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... on, um, heart strings and purse strings. You know, if we manipulate you into a slightly sadder mood, you'll pay more.
- LFLex Fridman
Yeah.
- RPRosalind Picard
Right? You'll pay more to change your situation. You'll pay more for something you don't even need to make yourself feel better.
- LFLex Fridman
Yeah.
- RPRosalind Picard
So, you know, if they sound a little sad, well, maybe I don't wanna cheer them up. Maybe first I wanna help them get something, a little shopping therapy, right, that helps them.
- LFLex Fridman
Which is really difficult for a company that's primarily funded on advertisement, so they are encouraged to, uh-
- RPRosalind Picard
Yep.
- LFLex Fridman
... to get, to get you to-
- RPRosalind Picard
To offer you products. Or Amazon's primarily funded o- on you buying things from their store. So, I think we should be, uh, you know, maybe we need regulation in the future to put a little bit of a wall between these agents that have access to our emotion and agents that wanna sell us stuff. Maybe there needs to be a little bit more of a firewall in between those.
- 25:38 – 30:32
Designing the ‘objective function’: helpful assistant vs button-pushing agent
- LFLex Fridman
So, maybe digging in a little bit on the interaction with Alexa, you mentioned, of course, a really serious concern about, like, recognizing emotion if somebody is, is speaking of suicide or depression and so on. But what about the actual interaction itself? Do you think-
- RPRosalind Picard
Mm.
- LFLex Fridman
... uh, so if I, if I, you know, you mentioned Clippy in being annoying. What is the objective function we're trying to optimize? Is it minimize annoyingness or minimize, uh-
- RPRosalind Picard
(laughs) Yeah.
- LFLex Fridman
... or maximize happiness? Well, but if we look at human-to-human relations, I think that push and pull, the tension, the dance, you know, the annoying, the flaws, that's what makes it fun. So, is there, um-
- RPRosalind Picard
Yeah. Yeah.
- LFLex Fridman
Is, is there a room for ... Like, what, what is the objective function?
- RPRosalind Picard
I mean, there are times when you wanna, you wanna have a little push and pull. Think-
- LFLex Fridman
Yeah.
- RPRosalind Picard
... think of kids sparring, right?
- LFLex Fridman
Right. Exactly.
- RPRosalind Picard
You know, I see my sons, and they, one of them wants to provoke the other to be upset, and that's fun. And it's actually healthy to learn where your limits are, to learn how to self-regulate. You can imagine a game where it's trying to make you mad, and you're trying to show self-control. And so if we're doing a AI-human interaction that's helping build resilience and self-control, whether it's to learn how to not be a bully or how to turn the other cheek or how to deal with an abusive person in your life, then you might need an AI that pushes your buttons.
- LFLex Fridman
Right.
- RPRosalind Picard
Right? Um, but in general, do you want an AI (laughs) that pushes your buttons? Mm, probably depends on your personality.
- LFLex Fridman
(laughs)
- RPRosalind Picard
I don't. I want one that's respectful, that is there to serve me-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... and that is there to, uh, extend my ability to do things. I'm not looking for a rival. I'm looking (laughs) for a helper. And that's the kind of AI I'd put my money on.
- LFLex Fridman
Your sense is for the m- majority of people in the world, in order to have a rich experience, that's what they're looking for as well. So they're not look- if you look at the movie Her, spoiler alert, I, I believe the, the program, the, the woman in the movie Her leaves the-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... person-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... for somebody else.
- RPRosalind Picard
Right. Right.
- LFLex Fridman
Says they don't wanna be dating anymore. Right? Like, w- a- g- do you s- y- your sense is if Alexa said, "You know what? I'm actually had enough of you, uh, for a while, so I'm gonna shut myself off," you don't see that as o-
- RPRosalind Picard
I'd, I'd say, "You're trash 'cause I paid for you," right?
- LFLex Fridman
Okay.
- RPRosalind Picard
You, you, we, um, we've gotta remember, and th- and this is where this blending human/AI as if we're equals is really deceptive, because AI is something, at the end of the day, that my students and I are making in the lab. And we're choosing what it's allowed to say, when it's allowed to speak, what it's allowed to listen to, what it's allowed to act on given the inputs that we choose to expose it to, what outputs it's allowed to have. Uh, it's all something made by a human. And if we wanna make something that makes our lives miserable, fine. I, I wouldn't invest in it as a business, uh, you know, unless it's just there for self-regulation training. But I think we, you know, we need to think about what kind of future we want. And actually, your question, I really like the, um, what is the objective function? Is it to calm people down? Um, sometimes. Is it to always make people happy and calm them down? Well, there was a book about that, right? The Brave New World, you know, make everybody happy, take your soma if you're unhappy, take your happy pill. And if you refuse to take your happy pill, well, we'll, we'll threaten you by sending you to Iceland, uh, to live there.
- LFLex Fridman
(laughs)
- RPRosalind Picard
I lived in Iceland three years. It's a great place. Don't take your soma.
- 30:32 – 34:54
Expressed vs felt emotion: what cameras can infer (even from a ‘poker face’)
- LFLex Fridman
So, in, in this process of, um, empowering people and sensing people and, uh-... what is your sense on emotion in terms of recognizing emotion? The difference between emotion that is shown and emotion that is felt? So, uh, yeah, uh, y- yeah, emotion that is expressed on the surface through your face, your body and, uh, various other things, and what's actually going on deep inside on the biological level, on the neuroscience level, or some kind of cognitive level.
- RPRosalind Picard
Yeah. Yeah. Whoa, no easy questions here. (laughs)
- LFLex Fridman
Well, it's like, yeah, I'm sure there's no-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... an- there's no definitive answer, but what's your sense? How far can we get by just looking at the face? Can you get-
- RPRosalind Picard
We're very limited when we just look at the face, but we can get further than most people think we can get. People think, "Hey, I have a great poker face, therefore all you're ever gonna get from me is neutral." Well, that's naive. We can read with the ordinary camera on your laptop or on your phone, we can read from a neutral face if your heart is racing. We can read from a neutral face if your breathing is becoming irregular and showing signs of stress. We can read, uh, under some conditions that maybe I won't give you details on, um, how your heart rate variability power is changing. That could be a sign of stress even when your heart rate is not necessarily accelerating. So-
- LFLex Fridman
I'm sorry, from physio sensors or from the face?
- RPRosalind Picard
From the color changes-
- LFLex Fridman
The color changes?
- RPRosalind Picard
... that you cannot even see, but the camera can see.
- LFLex Fridman
Mm-hmm. That's amazing. So, so you can get a lot of signal, but-
- RPRosalind Picard
So we get things people can't see-
- LFLex Fridman
Right.
- RPRosalind Picard
... using a regular camera. And from that, we can tell things about your stress. So if, um, you were just sitting there with a blank face thinking, "Nobody can read my emotion," well, you're wrong.
- LFLex Fridman
Right. So, so that's really interesting. But that's from sort of visual information from the face that's almost like cheating your way to th- the physiological state of the body by being very clever with what you can do with vision.
- RPRosalind Picard
With, with signal processing. Yeah.
- LFLex Fridman
With signal processing.
- RPRosalind Picard
Yeah.
- LFLex Fridman
So that's, uh, really impressive. But if you just look at the stuff we humans can see, uh, the-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... the poker f- the smile, the smirks, the- the subtle, all the facial expressions.
- RPRosalind Picard
Right. So then you can hide that on your face for a limited amount of time. Now, if you, if you're just going in for a brief interview and you're hiding it, that's pretty easy for most people. If you are, however, surveilled constantly everywhere you go, then it's gonna say, "Gee, you know, Lex used to smile a lot, and now I'm not seeing so many smiles." And, "Roz used to, um, you know, laugh a lot and smile a lot very spontaneously, and now I'm only seeing these not so spontaneous looking smiles."
- LFLex Fridman
Yeah.
- RPRosalind Picard
"And only when she's asked these questions." You know, that's... something's changed here.
- LFLex Fridman
Probably not getting enough sleep is part of it. (laughs)
- RPRosalind Picard
Um, we could look at that too. So, now, I have to be a little careful too. When I say we, we... you think we can't read your emotion and we can, it's not that binary. What we're reading is more some physiological changes that relate to the, your activation.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Now, that doesn't mean that we know everything about how you feel. In fact, we still know very little about how you feel. Your thoughts are still private. Your, uh, nuanced feelings are still completely private. We can't read any of that. So, there's some relief that we can't read that, even brain imaging can't read that, um, wearables can't read that. However, as we read your body state changes, and we know what's going on in your environment, and we look at patterns of those over time, we can start to, uh, make some inferences about what you might be feeling. And that is where it's not just the momentary feeling, but it's more your stance toward things, and that could actually be a little bit more scary with certain kinds of, uh, governmental, uh, control freak people who want to know more about, are you on their team or are you not?
- LFLex Fridman
And getting that information through over time, so you're saying there's a lot of signal-
- RPRosalind Picard
Mm-hmm.
- 34:54 – 39:00
Best modalities and why wearables + phones can forecast tomorrow’s mood and stress
- LFLex Fridman
So you've done a lot of exciting work, both in computer vision and physiological stuff like wearables. What do you think is the best modality for... what's the, uh, the best window into the emotional soul? Like, is it the face?
- RPRosalind Picard
(laughs)
- LFLex Fridman
Is it the voice?
- RPRosalind Picard
Depends what you wanna know.
- LFLex Fridman
Is it text? The body?
- RPRosalind Picard
It depends what you wanna know.
- LFLex Fridman
It depends.
- RPRosalind Picard
It depends what you wanna know. Everything is informative. Everything we do is informative.
- LFLex Fridman
So for health and wellbeing and things like that, do you find the wearable physio techn- so me- measuring physiological signals is the best for, um, health-based stuff?
- RPRosalind Picard
It... So here, I'm gonna answer empirically with data and studies we've been doing. We've been doing studies, now these are currently running with lots of different kinds of people, but where we've published data, uh, and I can speak publicly to it, the data are limited right now to New England college students. So that's a small group. (laughs) Uh, among New England college students, when they are wearing a wearable, uh, like, like the Empatic Embrace here that's measuring skin conductance, movement, temperature, and when they are, uh, using a smartphone that is collecting their time of day of when they're texting, who they're texting, their movement around it, their GPS, the weather information based upon their location, uh, and when it's using machine learning and putting all of that together, and looking not just at right now, but looking at your rhythm of behaviors over about a week. When we look at that, we are very accurate at forecasting tomorrow's stress, mood, and happy/sad mood, and, uh, health. And when we look at which pieces of that are most useful...... first of all, if you have all the pieces, you get the best results.
- LFLex Fridman
Right.
- RPRosalind Picard
If you have only the wearable, you get the next-best results. And that's still better than 80% accurate at forecasting tomorrow's levels.
- LFLex Fridman
Isn't that exciting? Because the wearable stuff with physiological s- information, it feels like it violates privacy less than the non-contact, face-based methods.
- RPRosalind Picard
Yeah. It's, it's interesting. I think what people sometimes don't r- you know ... It's funny, in the early days, people would say, "Oh, wearing something or giving blood is invasive," right?
- LFLex Fridman
Right.
- RPRosalind Picard
Whereas a camera is less invasive 'cause it's not touching you. I think on the contrary, the things that are not touching you are maybe the scariest because you don't know when they're on or off.
- LFLex Fridman
Right.
- RPRosalind Picard
And you don't know when pe- and you don't know who's behind it. Right? A wearable, uh, depending upon what's happening to the data on it, if it's just stored locally or if it's streaming, and what it is being attached to, uh, in, in a sense, you have the most control over it because it's also very easy to just-
- LFLex Fridman
You can take it off.
- RPRosalind Picard
... take it off, right? Now it's not sensing me. So if I'm uncomfortable with what it's sensing, now I'm free.
- LFLex Fridman
Yeah.
- RPRosalind Picard
Right? If I'm comfortable with what it's sensing, then ... And I happen to know everything about this one and what it's doing with it, so I'm quite comfortable with it. Uh, then I'm, you know, I have control. I'm comfortable. Control is one of the biggest factors for an individual in reducing their stress. If I have control over it, if I know all there is to know about it, then my stress is a lot lower, and I'm making an informed choice about whether to wear it or not, or when to wear it or not. I want to wear it sometimes, maybe not others.
- LFLex Fridman
Right. So that control. Yeah. I'm, I'm with you. That control. E- even if, uh ... Yeah, the ability to t- turn it off, uh, is-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... is a really empowering thing.
- RPRosalind Picard
It's, it's huge. And we need to maybe, you know, if, if there's regulations, maybe that's number one to protect, is people's ability to, uh, as easy to opt out as to opt in.
- 39:00 – 44:31
From stress signals to seizure detection: Empatica Embrace, SUDEP, and deep brain mapping
- LFLex Fridman
Right. So you've, uh, you studied a bit of neuroscience as well. How, how have looking at our own minds, sort of the, sort of the biological stuff or the, uh, the neurobiological, the neuroscience look at the signals in our brain, uh, helped you understand the problem in the approach of effective computing? So ...
- RPRosalind Picard
Hmm. Originally, I was a computer architect, and I was building, uh, hardware and computer designs. And I wanted to build ones that worked like the brain. So I've been studying the brain as long as I've been studying how to build computers.
- LFLex Fridman
Have you figured out anything yet? (laughs)
- RPRosalind Picard
(laughs) Very little. Uh, it's, it's so amazing. You know, they used to think, like, "Oh, if you remove this chunk of the brain-"
- LFLex Fridman
Right.
- RPRosalind Picard
"... and you find this function goes away, well, that's the part of the brain that did it." And then later they realized, "If you remove this other chunk of the brain, that function comes back," and-
- LFLex Fridman
Yeah.
- RPRosalind Picard
... "Oh no, we really don't understand it."
- LFLex Fridman
Yeah.
- RPRosalind Picard
Uh, brains are so interesting and changing all the time, and able to change in ways that s- will probably continue to surprise us. When we were measuring stress, you may know the story where we found an unusual big skin conductance pattern on one wrist in one of our kids with autism. And in trying to figure out how on earth you could be stressed on one wrist and not the other, like how can you get sweaty on one wrist, right? When you, when you get stressed with that sympathetic fight or flight response. Like you, you kind of should, like, sweat more in some places than others, but not more on one wrist than the other. That didn't make any sense. We learned that what had actually happened was a part of his brain had unusual electrical activity-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... and that caused an unusually large sweat response on one wrist and not the other. And since then, we've learned that seizures cause this unusual electrical activity, and depending where the seizure is, if it's in one place and it's staying there, you can have a big electrical response we can pick up with a wearable at one part of the body. You can also have a seizure that spreads over the whole brain, generalized grand mal seizure, and that response spreads and we can pick it up pretty much anywhere. As we learned this and then later built, um, Embrace that's now FDA cleared for seizure detection, uh, we have also built relationships with some of the most amazing doctors in the world who not only help people with unusual brain activity or epilepsy, but some of them are also surgeons and they're going in and they're implanting electrodes, not just to momentarily read the strange patterns of brain activity that we'd like to see return to normal, um, but also to read out continuously what's happening in some of these deep regions of the brain during most of life when these patients are not seizing. Most of the time, they're not seizing. Most of the time, they're fine. And so we are now working on mapping those deep brain regions that you can't even usually get with, uh, EEG scalp electrodes, uh, 'cause the changes deep inside don't reach the surface. Uh, but interesting, when some of those regions are activated, we see a big skin conductance response. Who would've thunk it, right? Like, nothing here, but something here. In fact, right after seizures that we think are the most dangerous ones, um, that precede what's called SUDEP, sudden unexpected death in epilepsy, uh, there's a period where the brainwaves go flat and it looks like the person's brain has stopped, but it hasn't. Uh, the activity has, has gone deep into a region that can make the cortical activity look flat, like a quick shutdown signal here.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
Um, it can unfortunately cause breathing to stop if it progresses long enough. Before that happens, we see a big skin conductance response in the data that we have. The longer this flattening, the bigger our response here.So we have been trying to learn, you know, initially like, why, w- why are we getting a big response here when there's nothing here? Well, it turns out there's something much deeper. So we can now go inside the brains of some of these individuals, fabulous people who usually aren't seizing, and get this data and start to map it. So that's active research that we're doing right now with, with top medical partners.
- LFLex Fridman
So this, this wearable sensor that's looking at skin conductance can capture sort of the ripples of the complexity of what's going on in our brain? So you c- this, this little device is, you have a hope that you can start to get the signal from the...
- RPRosalind Picard
Mm-hmm.
- LFLex Fridman
... from the interesting things happening in the brain?
- RPRosalind Picard
Yeah. We've, uh, we've already published the strong correlations between the size of this response and the flattening that happens afterwards, and, uh, unfortunately also in a real SUDEP case where the patient died because the, uh... Well, we don't know why. We don't know if somebody was there, it would have definitely pr- prevented it. But we know that most SUDEPs happen when the person's alone. And, uh, in this case-
- LFLex Fridman
What's a SUDEP? I'm sorry.
- RPRosalind Picard
A, a SUDEP is an acronym, S-U-D-E-P.
- LFLex Fridman
Okay.
- RPRosalind Picard
And it stands for, uh, the number two cause of years of life lost actually among all neurological disorders. Um, stroke is number one. SUDEP is number two, but most people haven't heard of it. Actually, I'll plug my TED Talk, it's on the front page of TED right now, uh, that talks about this. And, uh, we hope to change that. I hope everybody who's heard of SIDS and stroke will now hear of SUDEP, 'cause we think in most cases it's preventable if people take their meds and aren't alone when they have a seizure. Uh, not guaranteed to be preventable, there are some exceptions, uh, but we think most cases probably are.
- 44:31 – 48:31
Why FDA clearance is ‘agonizing’: safety, opacity, and innovation friction
- LFLex Fridman
So you have this embrace now in the version two wristband, right? Uh, for epilepsy management. That's the, the one that's FDA approved?
- RPRosalind Picard
Yes.
- LFLex Fridman
And which is kind of a cred-
- RPRosalind Picard
FDA cleared they say.
- LFLex Fridman
Cleared.
- RPRosalind Picard
Yeah.
- LFLex Fridman
Sorry if-
- RPRosalind Picard
No, it's okay. It's-
- LFLex Fridman
(laughs)
- RPRosalind Picard
It, it essentially means it's approved for marketing.
- LFLex Fridman
Got it. Just a, a side note, how difficult is that to do? It's essentially getting FDA approval for com-
- RPRosalind Picard
It's agonizing. (laughs)
- LFLex Fridman
... computer science technology.
- RPRosalind Picard
It's so agonizing. It's much harder than publishing multiple papers in top medical journals. Yeah. We have published peer reviewed top medical journal neurology best results, and that's not good enough for the FDA.
- LFLex Fridman
Is that system... So if we look at the peer review of medical journals, there's flaws, there's strengths. Is the FDA approval process, uh, how does it compare to the peer review process? Does it, uh, have the strength and, uh-
- RPRosalind Picard
I'll take peer review over FDA any day. (laughs)
- LFLex Fridman
But is that a good thing?
- RPRosalind Picard
Yeah.
- LFLex Fridman
Is that a good thing for FDA? Uh, you're saying, does it stop some amazing technology from getting through?
- RPRosalind Picard
Yeah, it does. The, the FDA performs a very important good role in keeping people safe. They keep things, they, they put you through tons of safety testing, and that's wonderful, and that's great. I'm all in favor of the safety testing. Sometimes they put you through additional testing that they don't have to explain why they put you through it, and you don't understand why you're going through it, and it doesn't make sense, and that's very frustrating. Uh, and maybe they have really good reasons, and they just would, it would do people a service to articulate those reasons.
- LFLex Fridman
Be more transparent.
- RPRosalind Picard
Be more transparent.
- LFLex Fridman
So as part of Empatica, we have sensors. So what kind of problems can we crack? What kind of things, uh, from seizures to autism, uh, to I think I've heard you mention depression-
- RPRosalind Picard
Uh-huh.
- LFLex Fridman
... what kind of things can we alleviate, can we detect? What's your hope of what, how we can make-
- RPRosalind Picard
Yeah.
- LFLex Fridman
... the world a better place with this wearable tech?
- RPRosalind Picard
I would really like to see my, you know, fellow brilliant researchers step back and say, you know, what are, what are the really hard problems that we don't know how to solve that come from people maybe we don't even see in our normal life because they're living in the poorer places, they're stuck on the bus, they're, they can't even afford the Uber or the Lyft or the data plan or all these other wonderful things we have that we keep improving on? Meanwhile, there's all these folks left behind in the world, and they're struggling with horrible diseases, with depression, with epilepsy, with diabetes, with just awful stuff that, uh, maybe a little more time and attention hanging out with them and learning what are their challenges in life, what are their needs, how do we help them have job skills, how do we help them have a hope and a future and a chance to have the great life that so many of us building technology have? And then how would that reshape the kinds of AI that we build? How would that reshape the new, you know, apps that we build, or the, maybe we need to focus on how to make things more low cost and green instead of thousand dollar phones. I mean, come on. You know? Why can't we be thinking more about things that do more with less for these folks? Quality of life is not related to the cost of your phone. You know? It's not something that... You know, it's been shown that what, about $75,000 of income and happiness is the same, okay? S- however, I can tell you, you get a lot of happiness from helping other people. You get a lot more than $75,000 buys. So how do we connect up the people who have real needs with the people who have the ability to build the future, and build the kind of future that truly improves the lives of all the people that are currently being left behind?
- 48:31 – 59:55
AI, embodiment, consciousness, and love—then a turn to faith, truth, and meaning
- LFLex Fridman
So let me return just briefly on a point maybe in movie Her.
- RPRosalind Picard
Mm-hmm.
- LFLex Fridman
So do you think if we look farther into the future, you said so much of the benefit from making our technology more, uh, uh, empathetic to us human beings would make them better tools, eh, empower us, make, make our lives better. But if we look farther into the future, do you think we'll ever create an AI system that we can fall in love with and loves us back on a level that is similar to human-to-human interaction?... like in the movie Her, or beyond?
- RPRosalind Picard
I think we can simulate it in ways that could, you know, sustain engagement for a while. Would it be as good as another person? I don't think so. For, if, if you're used to, like, good people. Uh, now, if you've just grown up with nothing but abuse and you can't stand human beings, can we do something that helps you there, that gives you something through a machine? Yeah. But that's pretty low bar, right, if you've only encountered pretty awful people. Uh, if you've encountered wonderful, amazing people, we're, we're nowhere near building anything like that. And I'm, I would not bet on building it. I would bet instead on building the kinds of AI that helps all, helps kind of raise all boats, that helps all people be better people, helps all people figure out if they're getting sick tomorrow, and helps give them what they need to stay well tomorrow. That's the kinda AI I wanna build, that improves human lives. Not the kinda AI that just walks on The Tonight Show and people go, "Wow, look how smart that is." You know? Really? Like, and then it goes back in a box, you know?
- LFLex Fridman
(laughs) So on that point, if we continue looking a little bit into the future, uh, do you think an AI that's empathetic and, uh, does, uh, improve our lives need to have a physical presence, a body? And even, let me, uh, cautiously say the C-word, consciousness, and, uh, even fear of mortality? So some of those human characteristics. Do you think it needs to have those aspects or can it remain simply a machine learning tool that learns from data of behavior that, uh, that learns to make us, based on previous patterns, uh, feel better? Or does it need those elements of consciousness and-
- RPRosalind Picard
It, it depends on your goals. If you're making a movie, it needs a body. It needs a gorgeous body. It needs to act like it has consciousness. It needs to act like it has emotion, right? Because that's what sells, that's what's gonna get me to show up and enjoy the movie, okay? Um, in real life, does it need all that? Well, if you've read Orson Scott Card, Ender's Game, Speaker for the Dead, you know, it could just be like a little voice in your earing, right? And you could have an intimate relationship, and it could get to know you, and it doesn't need, uh, to be a robot. Uh, but that doesn't make as compelling of a movie, right?
- LFLex Fridman
Right.
- RPRosalind Picard
I mean, we already think it's kinda weird when a guy looks like he's talking to himself on the train-
- LFLex Fridman
Right.
- RPRosalind Picard
... you know, even though it's earbuds. So we have these... Embodied is more powerful. Embodied, when you compare interactions with an embodied robot versus a video of a robot versus no robot, uh, the robot is more engaging. The robot gets our attention more. The robot, when you walk in your house, is more likely to get you to remember to do the things that you asked it to do-
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
... because it's kinda got a physical presence.
- LFLex Fridman
Mm-hmm.
- RPRosalind Picard
You can avoid it if you don't like it. (laughs) It could see you're avoiding it. There's a lot of power to being embodied. There will be embodied AIs. They have great power, and opportunity, and potential. There will also be AIs that aren't embodied, that just are little software assistants that help us with different things, that may get to know things about us. Will they be conscious? There will be attempts to program them to make them appear to be conscious. We can already write programs that make it look like, "Uh, what do you mean? Of course I'm aware that you're there," right? I mean, it's trivial to say stuff like that. It's, it's easy to fool people. Uh, but does it actually have conscious experience like we do? Nobody has a clue how to do that yet. That seems to be something that is beyond what any of us knows how to build now. Will it have to have that? Uh, I think you can get pretty far with a lot of stuff without it. Um, will we accord it rights? Well, that's more a political game than it is, uh, a question of real consciousness.
- LFLex Fridman
Yeah. Can you go to jail for turning off Alexa is what, uh, is the que- is the question for, uh, an, an election maybe a few, uh, decades from now.
- RPRosalind Picard
Well, Sophia robot's already been given rights as a citizen in Saudi Arabia, right? Even before women have, uh, full rights.
- LFLex Fridman
That's-
- RPRosalind Picard
Then the robot was still put back in the box to be shipped to the next, uh, place where it would get a paid appearance, right?
- LFLex Fridman
Yeah. It's, uh, dark and almost comedic, if not absurd. So I've heard you speak about your journey in finding faith.
- RPRosalind Picard
Sure. Uh-huh.
- LFLex Fridman
Uh, and how you discovered some wisdoms about life and beyond from reading the Bible.
- RPRosalind Picard
Mm-hmm.
- LFLex Fridman
And, and I've also heard you say that, you said scientists too often assume that nothing exists beyond what can be, uh, currently measured.
- RPRosalind Picard
Mm-hmm.
- LFLex Fridman
Uh, so it's-
- RPRosalind Picard
Yeah, materialism.
- LFLex Fridman
Materialism, right?
- RPRosalind Picard
And scientism, yeah. Mm-hmm.
- LFLex Fridman
So in some sense, this assumption enables the near-term scientific method, uh, uh, assuming that we can, um, uncover the mysteries of this world by the mechanisms of measurement that we currently have. Uh, but we easily forget that we've made this assumption, right?
- RPRosalind Picard
Mm-hmm.
Episode duration: 1:00:11
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode kq0VO1FqE6I
