Finding A Formula For Movie and Music Preferences There may be a way of accounting for taste, after all. Tim Westergren, founder of Internet radio service Pandora, and Reed Hastings, founder and CEO of Netflix, explain how their companies are trying to develop algorithms that predict whether someone will like a song or film.

Finding A Formula For Movie and Music Preferences

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

JOE PALCA, host:

This is Talk of the Nation: Science Friday from NPR News. I'm Joe Palca. Ira Flatow is away. OK, Mr. Director Man, hit it.

(Soundbite of song "Won't Back Down")

PALCA: OK. So, how do you from this...?

Mr. TOM PETTY: (Singing) Well, I won't back down...

PALCA: To this?

(Soundbite of song "Fortunate Son")

PALCA: To this?

(Soundbite of song "Keep on Rocking Me, Baby")

PALCA: And then, onto this?

(Soundbite of song "Mr. Jones")

Mr. ADAM DURITZ: (Singing) Shah la, la, la, la, la, la...

PALCA: Well, the answer is pretty simple: Just go to and type in Tom Petty. A computer sifts through hundreds of thousands of songs to pick tunes with similar traits. Whether it's classic soul qualities, major-key tonality or a shredding electric guitar solo, in theory, if you like the first one, you'll like all those similar songs. Now, that's in the song domain. Let's take a - let's look at the movie domain, Netflix. The online movie-subscription service, has been something along the same lines with movies. Rates some flicks you've seen, and Netflix will spit out some recommendations and - but they have been trying to upgrade the recommendation system.

So, a few years ago, they started up the Netflix Prize, a challenge to computer scientists to make those calculations work 10 percent better, meaning the movies it suggests for you would seem even more hand-picked and more of the thing you want to watch. And the prize? $1 million. So, if you think you can do it, it's probably worth trying. So far, no one has been able to crack it. This week, Netflix gave away $50,000 as a progress prize just to keep people, I guess, to keep them in the game, and that went to team that reached 9.44 percent improvement. Joining me to talk about the prize and how to get the computer to pick out the stuff you'd like are my guests. Reed Hastings is the co-founder, chairman and CEO of Netflix. He joins us on the phone from California. Welcome to the program, Mr. Hastings.

Mr. REED HASTINGS (Co-Founder, Chairman and CEO, Netflix, Inc.): Thank you, Joe.

PALCA: And Tim Westergren is the founder and chief strategy officer of Pandora. He joins us from the studios of KQED in San Francisco. Welcome to Science Friday, Mr. Westergren.

Mr. TIM WESTERGREN (Founder, Pandora): Thanks. Good afternoon.

PALCA: Good afternoon. So, I guess my first question is for you, Mr. Hastings. It amused me when I read this number, that you had a 9.44 percent improvement. What are we measuring here for improvement?

Mr. HASTINGS: It's reducing the - to win the million dollars...

PALCA: Right.

Mr. HASTINGS: You have to reduce the error in our predictions by 10 percent.

PALCA: I see.

Mr. HASTINGS: And the error is measured as the root mean squared error, RMS.

PALCA: RMS, yes, of course.

Mr. HASTINGS: And it's been reduced by 9.5 percent, as you said, in about two years. So, hopefully, with 9.5 percent confidence, I could say someone is going to get the million dollars this year.

PALCA: Wow. This year, well, we've only got a few weeks to go this year.

Mr. HASTINGS: Oh, this calendar year. I'm sorry. Over this - 2009.

PALCA: Over the next year. OK. Good. So, at least we've cleared up - so, I mean, OK, what's the game plan here? How does the computer do this? Is it - in movies, is it the leading person? Is it actors? Is it a style? I mean, what kinds of features go into predicting whether someone's going to like one movie, if they already said they liked a different movie?

Mr. HASTINGS: Well, you know, the Holy Grail, Joe, in this industry for a long time has been being able to use data to help people pick movies, pick books, pick music - all of these taste-based things more easily. Today, we all go by our friends, we go by critics online, and sometimes they're right, and sometimes they're wrong, and it's a bit haphazard. And our view is, if we can help people more reliably find movies that they think are great, then many people who watch more movies would be happier.


Mr. HASTINGS: And it turns out to be very challenging. People are just very individualistic.


Mr. HASTINGS: And...

PALCA: We like to think so.


(Soundbite of laughter)

Mr. HASTINGS: And so, you know, what we find is two people who can be identical in movie tastes on 100 movies, and then you make predictions for one based upon the taste of the other, and it goes wrong. And that's because, you know, one person has had a bicycle accident and the other hasn't, and you've got a movie that's got a bicycle accident that's traumatic, and they may react very differently for that.

PALCA: All right.

Mr. HASTINGS: So, you know, no one is ever going to be perfect.

PALCA: Right.

Mr. HASTINGS: And the question is, can we be better than the other ways that you choose movies? You know, for example, your friends and the reviews...

PALCA: Right.

Mr. HASTINGS: We think we're already there for most people, not for everyone. And to answer your question about how does it work, what we do is try to be agnostic about, well, movies like these or like that. And what we try to use is the actual user feedback. So, we ask our subscribers to rate movies on a one-through-five basis; did they love it, or did they hate it? And our average Netflix subscriber has rated over 200 movies. People really get into it. It gets kind of fun. Once they've rated all those movies, we then do all the statistical number-crunching to try to figure out what's the most likely movie that that person will love. And that's what we've gotten better and better at.

PALCA: I forgot to invite our listeners to join the conversation or at least to remind them that our number is 800-989-8255. That's 800-989-TALK, and you know, please feel free to join in. Hope we can get to some people before the segment ends. But I want to turn to you, Mr. Westergren, and see what's the - so, what's the experience with Pandora in terms of accuracy of predicting? Because I've used the service, and you know, you get a certain number - you know, you get a song and you get a thumbs-up or a thumbs-down. I guess it's simpler than a rating system. But how accurate are you in picking what people are going to like?

Mr. WESTERGREN: Well, I guess ultimately you measure that by usage. And you know, Pandora has about 20 million people that are registered for the service, and it's growing like a weed every day. And our basis for making recommendations is actually primarily based on the intrinsic musical properties of every song. So, we've actually spent about nine years now, a team of about 50 musicians, listening to songs one a time and analyzing each one of them along close to 400 musical attributes per song, kind of like musical DNA, and that's sort of the connective tissue that we used to form playlists.

PALCA: That seems - I mean, I - Bob Boylan at NPR is somebody who gets, you know, tens of thousands of cassettes - oh, not cassettes anymore - CDs or downloads or however he's getting the music. But that seems kind of labor-intensive.

Mr. WESTERGREN: Yeah, it's kind of crazy.

(Soundbite of laughter)

PALCA: It's working out for you, though.

Mr. WESTERGREN: Yeah. It's a kind of thing you'd expect to be hatched in, you know, in the academic setting.

PALCA: Uh-huh.

Mr. WESTERGREN: And it was - I mean, we spent a lot of years sort of in the wilderness building the Music Genome Project, which powers Pandora.

PALCA: Right.

Mr. WESTERGREN: And have actually only becomes a radio about three years ago.

PALCA: Wow. Wow, yes. Well, it's a - I - well, I think it's a really interesting service, and it's also very interesting way of finding out artist that you would never heard that turns out you're probably going to like. But let's take a call now and go to Roger in Denver, Colorado. Roger, welcome to Science Friday.

ROGER (Caller): Hi. Thank you.

PALCA: Sure.

ROGER: I was wondering how the (unintelligible) movie guys - how you would differentiate, on your 10-percent goal(ph), somebody who has already seen the movie?

PALCA: Huh. You mean, if they've seen and liked it versus if they haven't seen it and then they watch it and they've liked it.

ROGER: Yeah. If they've seen it before and it comes up on the list, do they reject that and say that doesn't count? Or is it included a part of the 10 percent?

PALCA: What about that, Mr. Hastings?

Mr. HASTINGS: In terms of the prize or the contest, we have a dataset which includes all of the ratings, and then we take out a million of those 100 million ratings and hide them.

(Soundbite of laughter)

Mr. HASTINGS: And the game is to guess what those ratings were, in other words, what the people actually said that they liked that film.

PALCA: I see.

Mr. HASTINGS: So, in that way, we're able to measure at least what they said they - how they liked it.

PALCA: See, for me, as someone who's been - you used the term the Music Genome Project, Mr. Westergren, and what I'm hearing here is the sort of an evolution in the Netflix sense of what happen with the Genome Project, as people started just asking simple questions of large databases and waiting to see what emerge from the data. So, I kind of like the parallel to the Genome Project in what they're doing.

Mr. WESTERGREN: You know...

Mr. HASTINGS: But - go ahead.

Mr. WESTERGREN: One of the motivations for launching the company was to try to create some system to help surface obscure bands, so, you know, all the working bands that nobody knows about that don't actually show up on lists of people who like this also like this. And so, you know, we thought if you could really understand the music, you'd be able to take those bands and put them in their right context. And we actually now - now, that we've been up for awhile - as you mentioned earlier, listeners, when they launch stations on the Pandora, can actually rate the songs as they hear them. And so, much like Reed and Netflix, we have accumulated this enormous amount of feedback from listeners, which is sort of their way of saying, you know, I launched my Tom Petty station, and this pick was a good pick or not a good pick. And so we have about two billion thumb-up and thumb-down feedback from listeners that we now actually have layered into the systems. So...

PALCA: Two billion with a B?



Mr. WESTERGREN: It's a - when people are able to influence their radio station, they go a little nutty. They love doing it.

PALCA: See, this is terrible, because what's that going to do to radio DJs?

Mr. HASTINGS: You know, I think that...

Mr. WESTERGREN: Well, you know, that's not your problem.

(Soundbite of laughter)

Mr. WESTERGREN: Yeah, the problem we're trying to solve, obviously, is to make radio better for you so it plays a kind of music you like, just the same way Reed's trying to make your queue of movies more personalized, and also, further to that, to create a radio station that can accommodate a much larger collection of artists, in our case, you know, over 60,000.

PALCA: Well, here's - I guess what I'd have to then say is, don't do this for news, because then I'm out of a job.

(Soundbite of laughter)

Mr. HASTINGS: That's called the wire services.

PALCA: Ugh, please, it's scary to think about. OK, let's take another call now and go to Ann in Mountain View, California. Ann, welcome to Science Friday.

(Soundbite of silence)

PALCA: Are you there? Did we lose you? Oh, too bad. OK, let's see who else we can do. Not much time before the break, but let's try Laurie in Oklahoma City, Oklahoma.

LAURIE (Caller): Hi. I'm - I've actually used Pandora quite a bit. I'm a Pandora addict, and I've had Netflix for about two years now. My question was geared towards the comment earlier about a traumatic bicycle accident. Would it be possible, eventually, for the consumer to put in a search engine, like, I don't want to see a movie with children dying or something like that? Be able to find out...

PALCA: To sort of fine tune the kinds of...

LAURIE: On the recommendations?

PALCA: Yeah.

LAURIE: Because I do use the recommendations quite a bit. I absolutely love that feature.

PALCA: Interesting question, Laurie. Mr. Hastings?

Mr. HASTINGS: Laurie, the answer is absolutely. We are, you know - really, all of us, Pandora - at the beginning of this great experiment about how to make the systems better and better. And you know, today you can do things like say, I don't want too much violence or I don't want too much sex, or you know, very large-scale items, but what we're working on is all of this micro-tagging...

PALCA: Mr. Hastings, I've got to interrupt. We'll come back to this, but we've got to take a short break, and when we do we'll come back and take more calls about Netflix and its selection process and Pandora as well. So, stay with us.

(Soundbite of music)

PALCA: This is Talk of the Nation from NPR News.

(Soundbite of music)

PALCA: From NPR News, this is Talk of the Nation: Science Friday. I'm Joe Palca. We're talking this hour about how computers - or computer algorithms - can calculate what films and music you'll enjoy. My guests are Tim Westergren - he's the founder and chief strategy officer of Pandora - and Reed Hastings is the co-founder, chairman and CEO of Netflix. And Mr. Hastings, you were just talking about the ability to give consumers more options in terms of the things they would choose when they're going to choose the kind of movie they'd like to be - have recommended to them.

Mr. HASTINGS: That's right. I mean, the Internet is continuing to get better. If you look at how much change there has been, you know, in the 15 years that we've had Web browsers, it's pretty astounding, and we're going to see continued improvement as we all learn from each other. We learn from Pandora, we learn from all the other companies involved in this, and we're all just getting better. So, within the next five years, I'll say you'll be able to specify, great movies, but no bike accidents, please.

PALCA: Yeah. I got it.

Mr. WESTERGREN: And for our version of that is we like, you know, music, but you know, more vocal counterpoint and a softer kick-drum.

(Soundbite of laughter)

PALCA: Good. We had a inquiry from one of our - one of the people following Science Friday on Twitter, which was basically the question of how mood affects people's selection in picking movies and, I suppose, picking films. I suppose, Mr. Westergren, that's not a big issue for you. It means whatever they happened to want at that particular moment, but do you have any data at all to speak to mood, Mr. Hastings?

Mr. HASTINGS: Yeah, I mean, people pick - you know, the classic one is one is, you know, the heavy duty, you know, indie film drama versus the light Adam Sandler comedy. They might both be great for you, but just at very different times and depending on what mood you're in.

PALCA: Yeah.

Mr. HASTINGS: And we let, of course, the users watch the movies when they want. So, in that way, once they have the DVDs at home from Netflix, they're able to decide, you know, what they want to watch, when they want it.

PALCA: See, that's the trouble is deciding, you know, in advance how you're going to feel when the movies arrive and knowing that you're going to need a particular kind, but you know, people are - people have obviously figured that out because Netflix is doing pretty well, as I understand it.

Mr. HASTINGS: That's right. What Netflix and Pandora are both doing in growing these Internet models it's very hard, but there is one model that's even harder. So, you know, we're doing people and movies, or people and music, and if you look at the dating services like, they're doing people and people, you know, and that's true - that makes our problem look relatively easy.

PALCA: Right.

Mr. WESTERGREN: I wouldn't feel so bad if you didn't like a movie, but if you didn't like your wife that might be a problem.

Mr. HASTINGS: Indeed.

(Soundbite of laughter)

PALCA: Let's take another call now and go to Brad in Detroit, Michigan. Brad, welcome to Science Friday.

BRAD (Caller): Hi. Thanks for taking my call.

PALCA: Sure.

BRAD: I hate to be critical of Pandora because that's a service I actually use quite a bit and I really like it, but I have to critique it just a little bit, and I guess I have a little bit of question in there, in that I've made certain channels - I've created - you create your radio channel based on a band or certain types of music that you like. Well, certain channels I've created I've tended to see I've only gotten four different bands within two hours, certain times. So, I was wondering if there's ways you're working to improve on that, or is it just because you just don't have enough music yet, or are there certain bands that you have trouble sort of getting into your database?

PALCA: Interesting question, Mr. Westergren.

Mr. WESTERGREN: Yeah, that's pretty unusual, but it's probably a reflection of the - your starting point might have been in a part of our genome that's kind of relatively sparsely populated. So, maybe the band is very unique sounding, but - so we have about 60,000 artists, about 600,000 songs, and we add somewhere in the neighborhood of 10,000 new songs a month. And I can definitely tell you that the one of our sort of primary objective is to make sure that you don't have too much repetition. So, we're working on it.

PALCA: OK, we have time for one final call, and that will go Sari in Berkley, California. Sari, welcome to Science Friday.

SARI (Caller): Thank you for taking my call.

PALCA: Sure.

SARI: I've used both services. Currently I'm really enjoying Pandora, but what I find is that in general, the Internet seems to lack the capabilities of a library search. So, I stopped using Netflix when I couldn't search on categories. For instance, I wanted to look for a mystery in the - put out in the last year. I had to remember - with Netflix, I had to remember the name of the director or an actor, but it just so simple, categories. So, instead of you predicting for us, how about letting us choose?

(Soundbite of laughter)

PALCA: That's an interesting point. Sari, thanks for that call. What about that, Mr. Hastings?

Mr. HASTINGS: Well, it's an excellent concept. Sari is giving me - at least we've got some ways people can browse by genres and various types, but to make it really easy to do recent mystery is a great idea.

PALCA: OK, well, I'm afraid that's where we're going to have to leave things, gentlemen. Thank you both very much for joining me. It's a fascinating issue and question and...

Mr. HASTINGS: Joe, I have to leave you with a non-computer-based suggestion.


Mr. HASTINGS: I just saw "Slumdog Billionaire" in the theaters, and everyone should go see it. It's great.

(Soundbite of laughter)


Mr. WESTERGREN: I second that.

PALCA: All right. Well, see? We're providing many services here. We teach you about science, and we give you movie advice.

(Soundbite of laughter)

PALCA: Thanks very much. Tim Westergren is the founder and chief strategy officer for Pandora, and Reed Hastings is the co-founder and chairman - I'm sorry - co-founder, chairman and CEO of Netflix.

Copyright © 2008 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.