Team Wins Siemens Prize For Speech Analyzer

NPR's Melissa Block talks to Matthew Fernandez, the high school junior from Oregon who, along with his partner, Akash Krishnan, won the team prize in the 2010 Siemens Competition in Math, Science and Technology. Fernandez and Krishnan built a computer algorithm that analyzes speech and figures out the emotional state of the speaker. Fernandez says the technology could improve service at call centers, where irate customers could be quickly recognized by the computer's phone menu and quickly put through to a human. It could also be used to help autistic children better understand the emotion of someone speaking to them.

Copyright © 2010 NPR. For personal, noncommercial use only. See Terms of Use. For other uses, prior permission required.

MELISSA BLOCK, host:

Two high school juniors from Oregon have won $100,000, taking the top team prize in the 2010 Siemens Competition in Math, Science and Technology.

Their research project has to do with computers that are able to analyze audio from a human voice and figure out the emotional state of the speaker.

The winners are Akash Krishnan and Matthew Fernandez. And Matthew joins us now from Portland, Oregon. Matthew, welcome and congratulations.

Mr. MATTHEW FERNANDEZ (Winner, 2010 Siemens Competition in Math, Science and Technology): Thank you.

BLOCK: Why don't you explain how this works, what you've come up with here with Akash?

Mr. FERNANDEZ: So we created a computer program that utilizes the frequencies and energies in voice and tries to recognize what emotions are being spoken. So we train our system using a bunch of audio that's already been defined as either actors speaking angrily or happily. And then when we get a new signal, we can compare it best with what we know about each of the energies and frequencies of the new signal.

BLOCK: And how accurate would say that is right now?

Mr. FERNANDEZ: Our system is atop of the world for recognizing emotion. We're probably more than 90 percent accurate at recognizing between just happy and sad but less accurate if you add in emotions like anger and fearful, things like that.

BLOCK: Can you explain how that works? I mean, how you can get a computer algorithm to decode human emotion?

Mr. FERNANDEZ: Yeah. Our system utilizes a bunch of information that we can extract. So we extract a total of 57 different audio features. And then using those features, we can extract them for a whole lot of data that's already been provided.

So one of our databases is from children interacting with a pet robot dog called Aibo, it's made by Sony - and these German children. And then utilizing this data we essentially train our system. So we teach it what energy and frequencies relate to each different emotion. And then when we get a new signal, we can say that matches up really well with anger or that matches well with sadness, and try to predict what emotion is being spoken.

BLOCK: Matthew, I read that you and Akash were inspired by the science-fiction movie "I, Robot." And I wanted to play a little clip. This is where we hear the robot, named Sonny, talking to police.

(Soundbite of movie, "I, Robot")

Mr. ALAN TUDYK (Actor): (as Sonny) My father tried to teach me human emotions. They are difficult.

BLOCK: So was that the starting off point for you?

Mr. FERNANDEZ: I would say yes. In fact, we were even inspired a separate section of the movie where a robot recognizes that its user is very afraid, and then it can protect its user. And we thought that's really handy. And we did a little bit of research and discovered our topic, and we've been really passionate and fixated on it for the last year.

BLOCK: What do you think the end result will be, now that you have this algorithm sort of working out now - what would the applications be for what you're doing?

Mr. FERNANDEZ: There are many applications for our research. I'd say anywhere where humans and computers are interacting with a voice component, our resources can be applied. So we can improve call centers. It's a great place where, if the computer - the costumer service can recognize that the user is especially anger at customer service and could send them to a representative, that's a quick and relatively simple application.

Something we're especially fixated on is creating a wristwatch-like device for autistic children that use a happy face, a sad face, angry face, to display the emotions that are being spoken around them. This could significantly improve the ability of autistic children to interact with their peers, and to understand the emotions that are being spoken around them.

BLOCK: Matthew, you're still a junior in high school. You still got college to come. But have you started thinking about your dream job, what you would want to do with your life?

Mr. FERNANDEZ: That's a tough question. I think my career will probably be rooted in science somehow. I expect that my dream job is probably kind of a place where I can really help people through doing good research or advise, you know, the governments or the U.N., say, on, you know, how to interact with robots and the concerns of the ethics with emotional robots. Things like that.

BLOCK: Well, Matthew, congratulations and all the best to you and your partner, too, Akash Krishnan.

Mr. FERNANDEZ: Thank you.

BLOCK: Matthew Fernandez is a junior at Oregon Episcopal School in Portland. And together with Akash Krishnan, he will share the top team prize of $100,000 in this year's Siemens Competition in Math, Science and Technology.

Copyright © 2010 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR. This transcript is provided for personal, noncommercial use only, pursuant to our Terms of Use. Any other use requires NPR's prior permission. Visit our permissions page for further information.

NPR transcripts are created on a rush deadline by a contractor for NPR, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the authoritative record of NPR's programming is the audio.

Comments

 

Please keep your community civil. All comments must follow the NPR.org Community rules and terms of use, and will be moderated prior to posting. NPR reserves the right to use the comments we receive, in whole or in part, and to use the commenter's name and location, in any medium. See also the Terms of Use, Privacy Policy and Community FAQ.