How Did That Ad Make You Feel? Ask A Computer Rosalind Picard specializes in something called affective computing. She designs technology that can measure and communicate human emotion. Her work started with autistic children, and from there, she moved on to using computers to assess people's emotional connections to brands.
NPR logo

How Did That Ad Make You Feel? Ask A Computer

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
How Did That Ad Make You Feel? Ask A Computer

How Did That Ad Make You Feel? Ask A Computer

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


At M.I.T.'s media lab, researchers are working to develop computers that understand our emotions. And the scientists have big plans for emotionally intelligent devices - new autism treatments, better customer service, perhaps even a method for making Super Bowl ads more amusing.

NPR's Steve Henn explains.

STEVE HENN, BYLINE: If you've ever called your bank...

UNIDENTIFIED MAN 1: Please hold while I connect you.

HENN: ...or your phone company.

UNIDENTIFIED WOMAN: ...about your wireless service.

HENN: ...or even your own office.

UNIDENTIFIED MAN 2: I'm sorry, but our switchboard is closed.

HENN: ...and slammed head-on into a voicemail system that's made you want to scream...

UNIDENTIFIED MAN 1: Or press three or press six.

HENN: ...then this story is for you.

ROSALIND PICARD: Pretty much all communication, all of our experience, has emotion. It's like weather. It's always there, so ignoring it is really rude. Right?

HENN: For years, Rosalind Picard, a professor at M.I.T.'s media lab, has been trying to teach computers to mind their manners, to pay attention to how we humans feel.

PICARD: Most people think, you know, technology doesn't need emotion but it does need to show respect for people's feelings. And you can't really show respect for people's feelings unless you can see people's feelings.

HENN: And to understand them. So, Picard spent her career doing things like teaching computers to recognize human facial expressions and the emotion tenor in our voices.

PICARD: In the early days, we were trying to teach computers to recognize emotion and we gathered a bunch of video examples, you know, from Disney. They're the masters of emotional animation.


PICARD: And we're showing these to computers with our machine learning systems, trying to teach the computer how to recognize, you know, happy, sad, angry and so forth. And a friend of mine comes in and he says, have you thought of using those for autism?

HENN: At the time, Picard knew almost nothing about it.

PICARD: And next thing I know, I'm learning about his brother and a whole lot of people and their experiences and challenges understanding emotion, especially processing it live in real time in a face-to-face interaction.

HENN: They realized, if they were going to teach computers to recognize real facial expressions in the real world, animated videos of Goofy and Mickey were not going to cut it.

Rana el Kaliouby at M.I.T. became Picard's partner.

RANA EL KALIOUBY: To actually do this, we needed a lot of data. We needed a lot of spontaneous data, people expressing emotions for real.

HENN: They needed to record and analyze thousands of facial expressions.

PICARD: So Rana and I decided we had to start Affectiva.

HENN: It turns out, marketers were eager to use their technology to test ads and those tests would give them the data they needed. Affectiva is now a thriving little start-up and one of its products, Affdex, is being used to test Super Bowl ads.

It's simple. You watch an ad online and if you opt in, Affdex watches you with your own webcam. Its algorithms will automatically recognize if you're smiling or smirking or shocked.

GRAHAM PAGE: Using just a webcam is a really appealing option to us because it's very simple.

HENN: Graham Page is at Millward Brown, a market research company. His clients include some of the largest advertisers on the planet. Last year, Affectiva used this technology to test this famous ad for Volkswagen.


HENN: Featuring a little boy dressed as Darth Vader using the force or trying to.

KALIOUBY: The more people watched it, the more enjoyable this ad became.

HENN: Rana el Kaliouby says even people who have seen this ad many times will still smile in anticipation of the punch line.

KALIOUBY: This is an essential element of a successful or a great ad that's going to go viral.

HENN: And this year, after the Super Bowl, Graham Page at Millward Brown will be testing more of his clients' ads. Anyone anywhere will be able to log on to Affectiva's website and watch commercials from the game.

PAGE: We'll be recording things like whether people are smiling, whether people are frowning, whether they're shocked and surprised, whether they're even paying attention.

HENN: The site will let you compare your reactions to the rest of the world and Picard says the amount of data they'll collect could be invaluable for advertisers and even for autism research.

Steve Henn, NPR News, Silicon Valley.

Copyright © 2012 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.