Supercomputer Takes On 'Jeopardy!' Champions
NEAL CONAN, host:
The long-running TV trivia game "Jeopardy!" involves a lot more than facts. A successful player must be able to assess ambiguity, nuances and puns at lighting speed, and this week host Alex Trebek introduced a most unusual contest.
(Soundbite of TV show, "Jeopardy!")
Mr. ALEX TREBEK (Host, "Jeopardy!"): You are about to witness what may prove to be an historic competition - an exhibition match pitting an IBM computer system against the two most celebrated and successful players in "Jeopardy!" history. Sounds like a lot of fun, doesn't it?
CONAN: After the first round, broadcast yesterday evening, the IBM computer nicknamed Watson is tied for first place with all-time money winner Brad Rutter. Seventy-four-time "Jeopardy!" champ Ken Jennings is currently in third. With two more rounds to go, the winner gets a cool million. If you saw the game and have questions, our phone number, 800-989-8255. Email us, firstname.lastname@example.org. You can also join the conversation on our website. That's at npr.org. Click on TALK OF THE NATION.
Ken Jennings joins us now from our bureau in New York.
Nice to have you back on TALK OF THE NATION, Ken.
Mr. KEN JENNINGS (Writer): Thanks for having me back, Neal. It's always a pleasure.
CONAN: And we know you're not going to reveal what happens tonight or tomorrow. Darn you.
Mr. JENNINGS: I'm sworn to secrecy. I have "Jeopardy!" snipers on the roof across the street right now.
CONAN: But I know that you've learned to evaluate your opposition over the years. What do you make of Watson?
Mr. JENNINGS: Watson is different than any player I've ever played before on "Jeopardy!" I remember back in 2004 during my first run on the show, you'd see a lot of players sort of psyche themselves out of the game. They get discouraged just getting introduced to me, you know?
Mr. JENNINGS: Watson doesn't know or care who I am. Watson never gets stage fright or nerves, never gets cocky, never gets discouraged, never makes mental mistakes. It's like The Terminator. It just keeps coming.
CONAN: And both you and Brad enjoyed some advantage as you continued on winning because of the fact that you became more relaxed. You weren't as flustered.
Mr. JENNINGS: That's right. And now, we're sort of, again, the opposite of playing, you know, a super intelligent computer whose mechanical thumb next to you can hear clicking away in sort of a creepy, insectoid(ph) way.
(Soundbite of laughter)
Mr. JENNINGS: I was not feeling relaxed on "Jeopardy!" this time.
CONAN: And I wonder have you also given thought to - and these games, of course, were played some time ago, taped, and they're being broadcast -but did you, at the time, were, wait a minute, I'm getting psyched worrying about Watson. There's this other guy who's pretty good too.
Mr. JENNINGS: Exactly. It's the worst of both worlds, you know? The ideal scenario would be to have a human versus a computer, or maybe a computer versus a very good human and a lousy "Jeopardy!" player. I don't know if you saw Wolf Blitzer on the show, but I'd like to have Wolf back.
(Soundbite of laughter)
Mr. JENNINGS: Unfortunately, Brad Rutter is very good. He beat me very badly at "Jeopardy!" last time we played. The guy has never been beaten, in fact, on "Jeopardy!" by a man or a machine. So this is a very tall order. I'm looking forward to the rematch with him as well.
CONAN: And what do you think you've learned about computer programs as you play against Watson?
Mr. JENNINGS: I'm a computer programmer myself, and I think a lot of people are a little bit naive about what the potential for computers are. You know, they sort of think this "Star Trek" level of computer already exists where we can ask it questions and it will do our bidding and understand what we mean.
In fact, computers are terrible with that kind of thing. A computer that could understand natural languages is always thought to be - thought to have been decades away. So the fact that IBM does have this system that understands, as you say, not just context but things like nuance and humor and the all the double meanings that make "Jeopardy!" clue work, this is a huge breakthrough. We're seeing something new here.
CONAN: In general, in fact, if you ask a computer a question, you get not an answer. You get a long list of possibilities.
Mr. JENNINGS: And Watson does the same thing. If you watched the show last night, you'll see some of those possibilities in the bottom of your screen. You sort of get something you never get from a human player, which is a peek inside the brain, and it's very good at using its past success rating to weigh those possibilities, to say which one do I feel the most sure about and am I sure enough to buzz?
CONAN: And it also made some mistakes, and I think if you had been reading about the contest, you probably know about this one. Host Alex Trebek asked a question that stumped not only you, Ken, but also Watson. Let's listen.
(Soundbite of TV show, "Jeopardy!")
Mr. TREBEK: The first modern crossword puzzle is published and Oreo cookies are introduced. Ken?
Mr. JENNINGS: What are the '20s?
Mr. TREBEK: No. Watson?
WATSON: What is 1920s.
Mr. TREBEK: No. Ken said that. Brad?
Mr. BARD RUTTER: What is the 19teens.
Mr. TREBEK: Yes, the 1910s. Correct.
CONAN: And that's an interesting mistake. It was so certain of its answer that it was going to repeat it no matter what.
(Soundbite of laughter)
Mr. JENNINGS: That's sort of an interesting glitch that I think we knew about Watson going in. It's blind and deaf to what's going on in the studio. All it knows is what gets fed to it electronically, which is the clues and then the correct answer, once that's been played. So it had no idea what I was saying. It's a closed box. It's interesting.
CONAN: You've sat in the - stood in the contestant area with all sorts of people. There is exactly what in the middle between you and your fellow human?
Mr. JENNINGS: The computer is not physically there. One problem with Watson is it's huge. The game was played at an IBM lab precisely because IBM - or Watson doesn't travel. It's the size of a small RV. It's the whole room. It's the only "Jeopardy!" contestant I've ever been inside.
(Soundbite of laughter)
Mr. JENNINGS: So what you see at the podium is not himself - itself. I should say itself. The tendency to say him is overwhelming.
Mr. JENNINGS: But it's an avatar. It's a flat screen with a picture on it and, again, this little mechanical thumb. And that's all its there. It's sort of looks like the monolith on 2011 looming above us.
CONAN: It does put you in mind, though, of the days when something like UNIVAC took up a whole room. And, well, and how we calculate time, that's not so long ago.
Mr. JENNINGS: It is interesting that to match the speed and the processing power of the human brain, they need to put 3,000 processor cores in parallel otherwise - I think the first time attempt that Watson could answer "Jeopardy!" clues, but it took hours to do so. It's amazing that we sometimes take it for granted, but the computational power we have between our ears is astounding.
CONAN: We're talking with "Jeopardy!" champion, Ken Jennings, as we speak, in third place in the computer challenge. Watson and his human rival are tied for first. Currently, there are two more rounds tonight and tomorrow. 800-989-8255. Email us: email@example.com.
We'll start with Rob(ph), and Rob is with us from Dover in Delaware.
ROB (Caller): Hi. Thank you for taking my call. I'm a truck driver so I didn't get to see the show last night. I saw in the paper, in USA Today, that Watson and the other fellow each had 5,000. You only had 2,000. Those are surprisingly low amounts at the end of a round. I'm curious as to what the reason was the final scores at the end of the day were so low.
Mr. JENNINGS: Part of the reason is a programming decision. This is actually a two-game final that's being spread over three days. The better to provide context for the match, you know, why this is such an AI breakthrough, provide IBM with some nice free advertising, et cetera. And so what you're seeing is the - just a single round, half of a normal "Jeopardy!" game got played last night. So this is like the first quarter of a basketball game.
ROB: I see. Okay.
Mr. JENNINGS: These are the early days.
ROB: Thank you.
CONAN: As we all know, in Double Jeopardy the scores can really change.
(Soundbite of laughter)
Mr. JENNINGS: That's right. That's what I hear.
ROB: I didn't realize that it was just the single "Jeopardy!" round, the first "Jeopardy!" round. Good luck, Ken.
Mr. JENNINGS: Thank you so much.
CONAN: Let's see if we can go next to - this is John(ph), John with us from Kennett Square in Pennsylvania.
JOHN (Caller): Hi and thanks for taking my call.
JOHN: I just had a question for Ken, if he thinks it's cheating that the machine was able to actually be fed the text file and instead of having to actually interpret it through audio.
Mr. JENNINGS: I guess Watson's defense of that is that the interesting challenge is answering the question, you know, things like recognizing text on a screen or understanding a suave Canadian game show host or not, peripheral to the problem. But you're right in that, you know, it certainly does have the speed advantages. It starts thinking about the clue, while Brad's and my eyes are still moving over the first words. And, again, it's got the advantage of computer circuitry timing its buzz, you know, with microsecond precision that no human can match.
So obviously, I don't think this is - I don't think it's cheating. I don't think it's unfair. These are things that computers are good at and, therefore, should be allowed to be good at. But it does make an uphill battle for the human race.
CONAN: So there are things that humans are good at too.
Mr. JENNINGS: That's right. I mean, we have our own advantages in things like understanding natural language. You know, Watson is not saying, hey, Ken has been speaking English like a native for 30-odd years and I'm only four years old. You know, that's - you play that as a species' hand you're dealt, I guess.
CONAN: John, thanks very much for the call. And here's an email question about that microcircuitry advantage from Steph(ph) in Sacramento. Doesn't Watson have an advantage with the buzzer?
Mr. JENNINGS: "Jeopardy!" devotees know how important the buzzer is. You know, on any given night, nearly all of the contestants know nearly all the answers. So it's all who has built up the best rhythm on the tricky "Jeopardy!" buzzer. But that only matters if you can come up with the correct answer.
It is astounding that Watson comes up with the correct answer as often as it does. But that said, with its advantage of speed that no human reflexes can match, Brad and I need to hope that it gets some answers wrong, that we find a Daily Double. Homo sapiens is going to need some luck, I think, on tonight and tomorrow's show.
CONAN: And this is an email from A.D. How does Watson decide how much to bet on Daily Doubles and on Final Jeopardy? This is where contestants, well, can bet a lot more than the question would ordinarily offer.
Mr. JENNINGS: Yeah. Wagering is often a place where you'll see very good "Jeopardy!" players fall down on the job. But I'm not expecting that out of Watson. That tends to be simple math. You know, this is game theory. And even the crappy computer I had at home in ninth grade could do math very fast, you know. So I assume it makes a mathematically, strategically perfect wager every time which a human can only dream of.
CONAN: What do you mean the mathematically perfect wager? I mean, if you're in third place and you hope to catch up to first, obviously, you're going to bet it all.
Mr. JENNINGS: Yeah. But it - unlike a human player, it knows exactly -in the past, I have gotten 68 percent of the answers correct in this category. In general, in "Jeopardy!" the optimal bet in this situation has been this. It's got thousands of past games in its head. It can run thousands of situations like the chess computer that beat Kasparov. And I'm sure that kind of thing is going on under the hood.
CONAN: If - have you been interested in its choice of categories? There are, what, six different categories per round and the - you - the -whoever wins wrings in the last correct answer gets to choose the next category. Is Watson running categories or is it picking favorites?
Mr. JENNINGS: Watson tends not to have weak spots the way a human would. You know, a human player, no matter how good will inevitably be a little better at a pet category than another. I'm good history but bad at opera, or whatever.
Mr. JENNINGS: Watson, with 15 trillion bytes of data in there, you know, effectively the size of the Library of Congress inside its memory banks, doesn't have weak spots like that. Mostly, its category selection is driven by trying to find Daily Doubles. It wants to find those clues where it can, you know, make its own wager and keep its opponents from doubling their scores. And that's not random. Those - you can analyze old data and find out where are those are most likely to be. Last night, I got very lucky and I believe found one the first time out of the box.
CONAN: And betting on the Daily Doubles and when you get to it Final Jeopardy, those are the critical wagers of any game.
Mr. JENNINGS: It's amazing how often it comes down to that, you know? A score - even the highest value clue on the board is only $2,000, whereas a Final Jeopardy wager could be five or 10 times that.
CONAN: We're talking with Ken Jennings, who won "Jeopardy!" 74 times in a row. He's now pitted against the all-time money winner on "Jeopardy!," Brad Rutter, and also an IBM computer named Watson. You're listening to TALK OF THE NATION from NPR News.
Three years IBM has devoted to its resources into developing this computer. What kind of a breakthrough do you think they have achieved?
Mr. JENNINGS: This is something new. This is not just something we're going to see on primetime game shows. You know, the next challenge for IBM is not "Wheel of Fortune." You know, they want to see this thing answering questions in the real world. Someday, when we have questions about medical diagnosis or tech support or our taxes, business analytics, whatever it is - and this day will come soon - we're going to be asking software quite a bit like Watson. That'll be part of the process. So to them, this is just the big, splashy debut of something that's going to change all of our lives.
CONAN: Let's go next to Robert(ph), Robert with us from Cincinnati.
ROBERT (Caller): A few years ago, there was a celebrated Final Jeopardy where the person who was going to win decided that he would make a wager that would allow for a three-way tie as sort of an altruistic gesture. Does Watson have the capacity for altruism?
Mr. JENNINGS: You're asking, does the machine love, I guess.
(Soundbite of laughter)
CONAN: I think there are more than a few plays written about this, Robert.
Mr. JENNINGS: My guess is that Watson plays to win. That's how I play. And I know the IBM engineers have told me they watched my style of play very carefully. I think I've had a target on my back for quite a while here.
ROBERT: Are you familiar with that particular episode, though? I'm just curious.
Mr. JENNINGS: I do. I don't think Watson would play for the tie. And my inclination is that I would not either. Maybe it's just - maybe it's personal pride. But I like to think it's the dignity of the species. I would like to see the big, bad machine go down in flames.
CONAN: The dignity of the species. And Robert, thanks very much for the call. Is Homo sapiens really in the contest here?
Mr. JENNINGS: Maybe it's just my own ego, but yeah, I feel like I've somehow, through some weird coincidence, been elected as the champion of carbon-based life on Earth against, you know, our new future oppressor.
CONAN: Silicon, yeah.
Mr. JENNINGS: And I would like to strike a blow while I have the chance.
CONAN: I, for one, welcome our robot overlords.
(Soundbite of laughter)
Mr. JENNINGS: You may have no choice, Neal.
CONAN: Let's go next to Todd(ph), Todd with us from Los Angeles.
TODD (Caller): Hi. My comment is I actually - I'm a devoted "Jeopardy!" fan and I watch every single night. I even had people over for the whole Watson introduction. And to be honest with you, after the first 15 minutes, we were pretty bored because the interaction between the contestants and Alex didn't have the same kind of fun that regular "Jeopardy!" has. And I probably will not watch the next two nights.
Mr. JENNINGS: I don't know, are you blaming Brad and myself for being to wooden, or is the problem the computer in the middle podium? You know, I think that is a possibility. You know, people have asked me, is "Jeopardy!" going to become all computers now? You know, why would we even continue if a computer can beat a human on "Jeopardy!"? And I always say that, well, you know, we still run track and field races, even though there's plenty of cars that can go faster than the fastest runner. We're interested in the human element, the psychology, the thrill of victory and the agony of defeat. So I would not look for, you know, a Mac versus PC battle on "Jeopardy!" anytime soon, for example.
CONAN: Todd, thanks very much. And what else are you going to watch at 7 o'clock. I think Todd has left, perhaps, to look up his TV Guide. The -Ken Jennings, though, I wonder what you and Brad talk about? There are breaks during the taping, when the commercials are supposed to be played that we're watching at home. What do you guys talk about?
Mr. JENNINGS: If I remember right, we were sort of marveling just the surreality of the experience. You know, here we are playing on a primetime game show against essentially a super-intelligent robot. You know, we are living in the future. Surely, the domed, underwater cities and the flying cars cannot be far away. It was a real highlight for both of us.
CONAN: Are all the questions in Esperanto this time?
(Soundbite of language)
Mr. JENNINGS: Someday.
CONAN: Someday. Let's go to Tom(ph), Tom with us from San Antonio.
TOM (Caller): Yeah. Hi, Ken, I had a question for you. I was wondering, did you secretly root for Brad when he was coming up with the answer, you know, to beat the computer or were you rooting for Watson? I mean, was there a feeling one way or the other?
Mr. JENNINGS: It's sort of funny, you know, because if I do have a "Jeopardy!" nemesis or archrival, super villain, whatever you want to call it, it would be Brad. We've played before and he's beaten me the last time we played, so I was looking forward to that rematch. But even so, I found myself wanting to see Brad win over the microchips. I guess it's a weird kind of xenophobia, where, you know, you got to be true to your own species no matter what, even if it's Brad Rutter.
CONAN: But ordinarily, at this point in the game, you'd be rooting for whoever, you know, whoever is, you know, furthest away, I suppose?
Mr. JENNINGS: It's true. You know, anybody who can take out the leader, that's who's on your side.
CONAN: All right. Tom, thanks very much for the call.
TOM: Thank you.
CONAN: And, Ken, obviously, again, these were taped a long time ago. What do you at 7 o'clock this week? Are you having friends over? Is everybody watching with you?
Mr. JENNINGS: You know, I'm New York promoting the show. So there has been no viewing party, which is disappointing. I know the VillageReach, the charity I played for back in Seattle, wanted me to come to their big, exciting viewing party and I had to say, no, I got to sit in New York doing media interviews.
Last night, I took my wife out for Valentine's Day, so I may have missed the show but I may have saved my marriage.
CONAN: Well, and a clue, tomorrow night, are you going to be hanging out at the Blarney Stone, knocking back a few and drowning your tears.
Mr. JENNINGS: Well, I can't specify whether they're tears of joy or tears of sorrow. But, yeah, I'd like to see the game.
CONAN: Ken Jennings, thanks very much for your time and good luck.
Mr. JENNINGS: Thank you so much. The future of the species is riding on it, so I appreciate it.
CONAN: Ken Jennings, one of "Jeopardy's!" all-time champs. He won 74 consecutive games. The IBM "Jeopardy!" challenge continues tonight. The winner will be crowned on Wednesday. Tomorrow, our regular visit with political junkie Ken Rudin, and our series on Oscar-nominated feature documentaries begins. Join us for that. This is TALK OF THE NATION from NPR News. I'm Neal Conan in Washington.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.