Kang Lee: Can Technology Detect Our Hidden Emotions? Developmental researcher Kang Lee says scientists can detect emotions by reading subtle physiological signals beneath the surface of our skin.

Kang Lee: Can Technology Detect Our Hidden Emotions?

  • Download
  • <iframe src="https://www.npr.org/player/embed/591879723/592157247" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


What's the - what's the connection between lying and our emotions?

KANG LEE: When we lie to cover up our own transgressions, you have this fear of being caught, and also you have this shame and guilt and also the delight they associate with that.

RAZ: This is Kang Lee.

LEE: I study developmental neuroscience with a special focus on emotion, lie telling, in children and adults.

RAZ: And Kang says specifically when kids lie, they learn pretty early on how to hide all of these emotions and control their facial expressions.

LEE: You have to manage all your facial expressions, your body language, the words you are going to choose to make your lie stick.

RAZ: So say I guess I should mention, Kang, that...

LEE: Yeah.

RAZ: ...Just like a couple days ago, the remote control on our Google Chrome thing was broken. Like, it was peeled back. Like, you had to use, like, you know, jaws of steel to peel it back. And I looked at my kids and I said, what happened? And they both looked at me and said I don't know. I said, you don't know what happened to this remote control? And they said, no. I said, did you break it? No. Did you touch it? Yes, but we didn't break it.

LEE: Exactly. Yeah, and they - when they answer these questions, they actually look right into your eyes, right? They did not actually avert their eyes, right?

RAZ: Right into my eyes - no, I didn't do it.

LEE: Yes, indeed. And then the majority of kids, when they lie, they actually look into your face and very seriously say, no. I'm talking about very young kids. These could be 2 years old or 3 years old. So they are very good at managing their facial expressions.


RAZ: Now, remember how Lisa Feldman Barrett just said that our ability to detect emotions in other people is unreliable and that a facial expression doesn't necessarily tell you much about how someone's feeling? Well, Kang believes there might be a way to detect emotions without using our eyes. Kang Lee explains his research from the TED stage.


LEE: We know that underneath our facial skin, there's a rich network of blood vessels. When we experience different emotions, our facial blood flow changes subtly. So by looking at facial blood flow changes, we can reveal people's hidden emotions. We have developed a new imaging technology we call transdermal optical imaging. To do so, we use a regular video camera to record people when people experience various hidden emotions. And then, using our image processing technology, we can extract transdermal images of facial blood flow changes. And using this technology, we can now reveal the hidden emotion associated with lying and therefore detect people's lies with an accuracy at about 85 percent.


RAZ: So let me see if I understand this correctly. You can train a video camera onto a human face, and then, with your technology, you can analyze facial blood flow, and then that indicates how a person is feeling with an accuracy rate of 85 percent when it comes to lying.

LEE: Yes.

RAZ: But can you do that with other emotions?

LEE: Well, right now, we can measure your emotion at about 93 percent accuracy to differentiate between three states - positive, neutral and negative. And then with regard to specific emotions, depends on the situations. For example, disgust - we can do about 89 percent accuracy. But some other emotions are not very good. Fear - fear is the most difficult one, and it's about 64 percent. But still, you know, we have a long way to go to be able to pick up information from your face to say, ah, you know, Guy now is - he's experiencing fear or something like that.


RAZ: In a minute, just how Kang Lee's technology works and whether we can actually keep our emotions private ever again. I'm Guy Raz, and you're listening to the TED Radio Hour from NPR.


RAZ: It's the TED Radio Hour from NPR. I'm Guy Raz. And on the show today, Decoding Our Emotions. And we were just hearing from psychologist Kang Lee describe a new technology that he's created in his lab by using something called transdermal optical imaging.

LEE: Yes, and the technology actually is very simple.

RAZ: So basically, it starts with our skin, which is translucent.

LEE: So when lights come into your face, it does not bounce back right away. So it actually penetrates to the deeper layers of our face.

RAZ: And just by using a regular video camera to record a face...

LEE: You know, the cameras on our iPhone is amazingly good.

RAZ: ...That video footage - when it's run through Kang's machine learning algorithms, it can pick up a lot of physiological clues under our skin.

LEE: So they actually can pick up these reflections from your face.

RAZ: But what are they actually picking up?

LEE: OK. So the vascular system and the heartbeat - heartbeat variability, which is extremely important to measure your stress. And then you have breathing. So these are very, very good indications of how much stress you are in. And they all come to modulate the blood flows on your whole body. And all these activities comes out as a symphony, basically, on your face. And from there, what we're doing right now is to basically decode it, like, decipher these secrets coming out of your face to know what's going on in your physiological system.

RAZ: And over time, Kang has found markers for what happens in our bodies when we're, say, disgusted or surprised or angry. And I know what you're thinking. We just heard Lisa Feldman Barrett say that universal emotions don't exist and that emotions vary based on culture and context. And for the most part, Kang, well, he says...

LEE: Yeah, so I agree with Lisa. Cognitive appraisal is a very, very important part of our emotional experiences. So, for example, the same physiologic activities but in two different situations would give rise to entirely different emotional experiences. So let's say you have short of breath, you know, your heart rate is pumping. One situation is you're fearful, but the other situation maybe you are in love. So - and this is all about you making this cognitive appraisal of the situation.

RAZ: So is that why your team is still at, like, 64 percent accuracy when it comes to fear? Because the physical markers, the physiological markers for fear are so much harder to distinguish with, like, falling in love or anger.

LEE: Yes. We tend to think about emotions as kind of natural, instinctive reactions. But rather, the brain plays a very important role to say, I'm falling in love. That's why my heart quickens. You know, I start to sweat. You know, my face turns red. On the other hand, you may say, I just feel angry. My heart quickens. My face turns red and my breathing become faster - something like that. But fortunately, though, we as humans tend to respond to a host of things in a very similar way. And because of this, then you say, OK, you know, these are the signatures of disgust. These are the signatures of joy.

RAZ: I mean, if, in fact, you can connect certain physiological signals to specific emotions, surely it has to be adjusted for cultural context - right? - because there are some cultures that we know of that don't experience sadness because they don't have the language. The word doesn't exist.

LEE: Totally. I mean, just - I grew up in China. So - and then I started to study psychology in college. And one of the words I came across was depression in English. And I had no idea what depression was because in China, in terms of diagnosis, it's called love sickness. So basically, the appraisal of these psychiatrists about your depression is not because your bio-chemical imbalance but just simply you cannot find a partner for life.

RAZ: Wow.

LEE: So just think about it. This is just one simple example. There are many, many other cultures that may have language for certain emotions but not for some others. So these differences must come to play.

RAZ: So essentially, the thinking is that, over time, as this technology gets better and better and as the appraisals really do match the physiological signals, you can adjust this for different cultures and different contexts?

LEE: Well, certainly. So what we're doing now, actually, is not only adjust according to your culture. We are adjusting according to you. Just imagine a few years down the road, there is a robot at home. And your robot basically is videotaping you and extracting physiological activities from you by remotely using our technology, for example. And then they figure out, you know, in these kinds of situations, oh, that's why you are happily reading books to your kids. This situation - you know, you get frustrated when you burn your toast or something like this, right? They learn from your past information about you. And then they can pretty much, much better about your personal emotional experiences.

RAZ: OK, so there could be in the future a robot in your house that's just constantly assessing you based on your face and what's going on behind your skin - and just that that could be a reality in the future?

LEE: Oh, yeah, definitely. It's in the very, very near future, not remote future.

RAZ: So I don't want that future. Go back. Go back to the past. Go back, future. It's scary. That scares me. I don't want a robot knowing my emotions.

LEE: Really? Because I thought this will be good - you know, I discovered the activities we can pick up are very, very useful for monitoring our health. For example, you know, we - by looking at the facial blood flow changes, we discover some people have arrhythmia. And we can tell them their blood pressures as well. And from there, we actually now can measure your stress very accurately. So this is, to me, very, very useful.

And then from there - I'm thinking about my parents, right? They're in China. So they live by themselves. And they are in the 90s, you know. And sometimes they become lonely. So I try to call them every day. But that's still only like 10, 15 minutes. But they want to sometimes have conversations with someone. So this is what I'm kind of envisioning - you know, creating a robot for my parents, so they can converse them about their inner emotions. And sometimes they may feel depressed a little bit, and then the robot can help them to lift their spirit up.


RAZ: That's Kang Lee. He's a professor at the University of Toronto. You can see his entire talk at ted.com.

Copyright © 2018 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.