IBM Computer Faces Off Against 'Jeopardy' Champs

IRA FLATOW, host:

You're listening to SCIENCE FRIDAY. I'm Ira Flatow.

Just keeping you abreast of any things that might be going on in Cairo now, it's evening. There's a celebration in Tahrir Square. The crowds are shouting: Egypt is free.

Egyptian President Hosni Mubarak stepped down from his post today, ceding all power to the military. Vice president and intelligence chief Omar Suleiman announced during evening prayers that Mr. Mubarak had passed all authority to a council of military leaders.

So you can stay with NPR News throughout the day. We'll be bringing you updates as they happen, as we talk about science on SCIENCE FRIDAY.

And in that theme, ever since John Henry raced a steam-powered hammer -he really did - we humans have been competing with machines, testing human powers against the powerful devices we humans create.

Next week is the next chapter in that story. This time, man versus machine in a trivia contest - not just any trivia contest, a special one. You know, it's an IBM supercomputer named Watson, and it's going to compete against champions on the game show "Jeopardy!" I wonder if Alex Trebek will be there. I guess he will be - he always is. He's going to have to be on that show.

IBM has taken on this challenge before. If you remember the IBM computer Deep Blue, that's the one that beat chess champ Gary Kasparov in 1997. Well, this is a different computer. This one is named, famously, Watson. IBM loves that name.

And it doesn't just have to know math, but Watson has to understand natural language, too. It has to understand the nuances of those questions. You know, sometimes they have little different meanings in the "Jeopardy!" questions when you watch them, yeah. It has to be able to decipher those little nuances and the irony. It's a much tougher challenge than just thinking, what, 20 steps ahead in chess, as the other chess programs did.

My guest has written a new book about the development of Watson. He's here to talk about it. Stephen Baker is former technology writer for BusinessWeek. He's the author of the new book "Final Jeopardy: Man Vs. Machine and the Quest to Know Everything." Welcome to SCIENCE FRIDAY.

Mr. STEPHEN BAKER (Former Technology Writer, BusinessWeek; Author, "Final Jeopardy: Man Vs. Machine and the Quest to Know Everything"): It's a pleasure to be here. Thanks.

FLATOW: Whose idea's was this? Was it IBM's idea or "Jeopardy!" idea?

Mr. BAKER: Oh, this is IBM's idea. They wanted to cast the spotlight on themselves and show that they could do really sexy technology.

(Soundbite of laughter)

FLATOW: Well, they had the history with the chess playing match, right?

Mr. BAKER: Right. They need to do this kind of thing because they're not like Apple and Google. They don't have stuff that people want. So they have to show that they can do really fun stuff so that they can attract, you know, great Ph.D.s to their programs.

FLATOW: Well, they have stuff that businesses want.

Mr. BAKER: They have stuff that businesses want, but it doesn't...

FLATOW: That's sexy (unintelligible).

Mr. BAKER: It's not as fun.

FLATOW: And it's not as fun. So the machine has a different challenge than just making chess moves. It has to actually understand the question that's being asked of it in typical "Jeopardy!" language.

Mr. BAKER: Right. Now, a lot of people compare, say oh, doesn't Google already do this? But when you are using Google, you type in two or three words that help - that orient the machine. You try to make it as clear as possible.

"Jeopardy!" is the opposite. That computer, Watson, has to understand really difficult clues, and then it has to go and find an answer that it can bet on, not just a page, an answer that it can bet on.

FLATOW: 1-800-989-8255 if you'd like to talk about the "Jeopardy!" challenger, with Watson. And also you can tweet us @scifri, @-S-C-I-F-R-I. You said it has to bet on something.

Mr. BAKER: That's what "Jeopardy!" is all about.

FLATOW: That's right because you have to - that's right, you risk the money or the total points that you have.

Mr. BAKER: Right.

FLATOW: And so the machine has to decide whether it's going to bet or not.

Mr. BAKER: It has to decide whether, in the circumstances of the game, where the game stands, it's worth the risk with its 73 percent confidence in Hosni Mubarak as the answer, whether it's going to risk betting that or not.

FLATOW: You know, there is a version on the Internet, and we have it up on our website, you can go to sciencefriday.com, and play the Internet version of the Watson "Jeopardy!" game. And I played it yesterday. It's fascinating. It's fascinating. And you can watch Watson calculate its odds, right?

But it has to get to a certain, almost to half, almost a 50-50 chance that it knows the answer, before it decides to bet, at least on the online game, right? It's .48 or something like that?

Mr. BAKER: Well, it changes. In the real game, it changes a lot, because if it has a big lead late in the game, it's going to take - it's just like a human. It'll take far less chances.

FLATOW: Right.

Mr. BAKER: And it'll bet the farm if it's way behind.

FLATOW: Now in the real "Jeopardy!," you watch people, they buzz in.

Mr. BAKER: Right.

FLATOW: And sometimes if they time that incorrectly, they may lose the opportunity to ask the question. Is that built into the computer game, also?

Mr. BAKER: Yes, but the computer works differently than people. People anticipate the end of Alex Trebek's sentence, at which point they can buzz. And Watson doesn't have that anticipation, but as soon as the light goes on, Watson, if it knows the answer and has confidence in it, buzzes almost instantly.

So it's really a challenge for humans to beat Watson to the buzzer.

FLATOW: Does it have a mechanical buzzer, like a finger?

Mr. BAKER: That was a point of controversy, because IBM originally didn't have one. They were basically sending email. Watson was buzzing in electronically. And the "Jeopardy!" people said, hey, wait a minute, this isn't fair. You guys have to build a finger for that machine, and it has to press the physical buzzer just like the humans.

And IBM said: Wait a minute. You're turning - we built a brain, and you're turning it into a robot. This is like a Frankenstein monster. They said: You're grafting human limitations on to our machine. But they obeyed.

FLATOW: No wonder they need a sexy device if they think like that.

(Soundbite of laughter)

FLATOW: But they did. So they realized - they would make an actual finger that would buzz in. So it's just like being a contestant, or more like.

Mr. BAKER: It's more like being a contestant. But when you're playing "Jeopardy!," if you buzz in a little early, you get penalized a quarter of a second, and you're going to lose the buzz to somebody else. Watson never gets penalized.

FLATOW: Well, that's not fair.

Mr. BAKER: Well, that's what some people might say.

FLATOW: Let's see what our listeners say, 1-800-989-8255. Let's go to Gardner(ph) in South Carolina. Hi, Gardner.

GARDNER (Caller): Hey, how are you doing? Big-time fan of the show. I've been listening for a long time.

FLATOW: Thank you.

GARDNER: My question is about the actual way that Watson searches for information. Does he search like an ID database, like on the Net, or does he have some sort of a, like, physical hard disk that just contains a wealth of information?

FLATOW: Yeah, can he search Wikipedia, something like that, for the answer?

Mr. BAKER: Well, Watson, like the human contestants, has to have all the data stored in - locally, in itself, in what would be its head. And so it has millions and millions of documents. And Watson really doesn't know anything, but when - after it deciphers the clue, it goes on a hunt and tries to come up with what appears to be, in everything that it finds, the best possible answer for that clue.

GARDNER: So it searches internally, like with a search term?

Mr. BAKER: It searches internally, and it has more than a hundred different algorithms, and each one searches the clue with a different method.

Some look for poetry. Some look for puzzles. Some look for historical facts or geography. And then they come back, and the computer has to decide which of this motley gang of clues has the best chance to be right.

FLATOW: Thanks for calling. But you know what I noticed when I played the game online, is that the nuance questions bog it down. Sometimes, you know, there are two-part answers to some of those things, two-part questions we have to take, you know, the ending of one word and put it on the beginning of another name, and you've got this compound sort of answer. It has trouble with that sort of thing.

Mr. BAKER: Right. I mean, that's the greatest challenge is the nuance of human language. That - but I find it amazing how well Watson does in that area, actually.

FLATOW: Is it true that your book is not finished at this point, there's an ending to it that has yet to be done?

Mr. BAKER: Well, the funny thing is you can get - the book is already out there as an ebook, and you can get the first 11 chapters as an ebook on whatever device you have.

And then the final chapter, I've already written it.

FLATOW: But?

Mr. BAKER: But it won't come into the electronic - it won't arrive as an update until after the match on Wednesday, and then the regular, actual hardback book comes into the stores on Thursday.

FLATOW: Do you know who won the match on...?

Mr. BAKER: I do.

FLATOW: There are only a couple people listening.

Mr. BAKER: Right.

FLATOW: Give us a little...

Mr. BAKER: I had to sign a document with so many clauses in it, it would make you dizzy trying to read it. In fact, I didn't. I just gave up.

FLATOW: If you just roll your eyebrows...

(Soundbite of laughter)

FLATOW: ...don't have to say a word. Were you surprised who won?

Mr. BAKER: Well, let me put it this way: I had seen Watson - because I've been working on this project for a year, and I've kind of been embedded in the IBM team. So I've been watching Watson go through scores of practice games. And what I see is that Watson is very capable of losing on any day. It's a very good player, but a Final Jeopardy can screw it up. A category can totally mystify it sometimes.

So I wasn't - I would not be surprised either way, but I was extremely interested, and I think it was fun.

FLATOW: Let's go to Michael(ph) in Cleveland Heights. Hi, Michael.

MICHAEL (Caller): Hey, how are you? I saw - two quick questions. They kind of associate - they're associated together. One, I saw a little bit of the making of this on PBS. It was intriguing. It was amazing.

Two things. One: Does Watson itself only know what the answer or, you know, what's on the blue and white screen, does it have a cam that sees it, or does it just respond to the answer as orated by Alex Trebek, and they had a different guy in the prelim.

And the second one is: How does this Watson deal with, say, like, metaphor, hyperbole, some of the little things that, you know, only come from the mind, imagination, as well as, like, you know, you stick in the emoticons and stuff in there, from, like, texting that they use in clues now?

Mr. BAKER: Okay. Watson gets electronic text, just like the scoreboard at Yankee Stadium or something.

FLATOW: So it's not scanning, like our eyes are scanning the screen?

Mr. BAKER: No. It gets an electronic feed of the words and then goes about the very difficult business of interpreting that sentence.

FLATOW: But I have to read it and then figure it out. If it's getting it in(ph) immediately, isn't that cheating a little bit?

Mr. BAKER: Well, you have to read it, but you, being a human, understand it very easily and intuitively. Watson - for Watson it's a totally foreign language, and it spends a lot of time, for it, like maybe half a second, puzzling out the meaning of the sentence. And sometimes it doesn't understand it, so I'd say humans have the advantage in that area.

FLATOW: Uh-huh.

Mr. BAKER: Now, what was his second question?

FLATOW: It's what we talked about before, the little nuances in the language...

Mr. BAKER: Oh, yeah.

FLATOW: ...it's hard - double meanings of words, things like...

Mr. BAKER: You talk about metaphors.

FLATOW: Yeah.

Mr. BAKER: If a metaphor exists in the popular - in popular language, then Watson can find that metaphor on numerous occasions and can come to understand what it means and put it into the right context. If it's a new metaphor, it's much more difficult for it, because Watson really doesn't think the way we do or understand the world the way we do.

FLATOW: Let's go to Bethel(ph) in Harrisonburg, Virginia. Hi.

BETHEL (Caller): Hi. I was just wondering how does Watson determine what to wager on Daily Doubles and Final Jeopardy, and how did scientists get him to determine the way that he does it?

Mr. BAKER: You know, I find it interesting people are calling it him.

(Soundbite of laughter)

Mr. BAKER: But, anyway, okay. The way that they prepared this computer for Daily Doubles was that they put it through, literally, millions of simulated games, and they calculated how various responses on Daily Doubles at every conceivable juncture of the game fared and influenced the outcome of the game.

And so Watson on the 33rd question, with each - with the score as it is, can figure out exactly what its odds are for winning the game and what's the precise Daily Double bet that improves its chances of winning.

FLATOW: Wow.

BETHEL: Okay.

FLATOW: Are you going to be watching, Bethel?

BETHEL: Most definitely. Yes. I also saw the PBS special and was very intrigued.

FLATOW: That's great. Thanks for calling. So it's going to be on - I'm talking with Stephen Baker, author of "Final Jeopardy: Man vs. Machine and the Quest to Know Everything."

Who are the - who's Watson taking on?

Mr. BAKER: Okay. Watson is taking on two legends of "Jeopardy." One much better known the other. Ken Jennings...

FLATOW: Sure.

Mr. BAKER: ...in 2004 became the first "Jeopardy" celebrity. He won 74 straight matches, transfixed the country for the entire summer.

FLATOW: I remember that. He took over the show.

Mr. BAKER: Right. He took over the show.

FLATOW: You tuned in just to see Ken Jennings.

Mr. BAKER: Right. And then - "Jeopardy," of course, is not dumb, and they brought Jennings back for a tournament of champions because they wanted to parade their star the next year. And they brought this guy named Brad Rutter, who had never lost a "Jeopardy" game. Unlike Jennings, he had to retire after he won five games, because until 2003 you could win a maximum of five. So he hadn't played nearly as many games, but in the tournament of champions, he killed Ken Jennings.

FLATOW: He ran the table.

Mr. BAKER: He ran the table.

FLATOW: And he wiped out Ken Jennings.

Mr. BAKER: He wiped him out, and so Rutter has never lost a "Jeopardy" game and his speed on the buzzer is legendary. And he attributes his speed on the buzzer to his childhood when he played Mario Brothers, Super Mario.

FLATOW: Sure. Sure.

Mr. BAKER: He thinks that button work really helped him on the "Jeopardy" buzzer.

FLATOW: I believe - we're talking with Stephen Baker, author of "Final Jeopardy: Man vs. Machine and the Quest to Know Everything" on SCIENCE FRIDAY from NPR. I'm Ira Flatow, and that - if you want to watch "Jeopardy," that will be on this Thursday, did you say?

Mr. BAKER: No. It's on Monday, Tuesday and Wednesday.

FLATOW: Oh, the whole...

Mr. BAKER: It starts...

FLATOW: It's a three-day tournament.

Mr. BAKER: Yes. Cancel your Valentine's Day dinner.

(Soundbite of laughter)

Mr. BAKER: Don't buy roses, just tune up - tune in to "Jeopardy," Monday, Tuesday and Wednesday.

FLATOW: And so they play in - they add up the score for the three days, or how does that work?

Mr. BAKER: Yeah. Well, it's actually, it's two games, and they've divided them over three games so they can put more information about how Watson works and more - frankly, more promotional information about IBM, so...

FLATOW: I believe it.

(Soundbite of laughter)

FLATOW: Is he - you said it's Watson - in real life, Watson was a he. Is Watson modeled on a human brain, or are they going to learn anything from this experience about making...

Mr. BAKER: Watson...

FLATOW: ...computers more human?

Mr. BAKER: Watson is not modeled on a human brain, and in most of what -for most of what we use our brains for, Watson is a total loser. You know, it doesn't know anything. It doesn't think the way we do. Watson has these algorithms that go out and search and search and do the statistics on knowledge and come back with answers, but it's a computational process. And it's an amazing one, and you'll see, it comes up with answers. But outside of the realm of digging through massive mountains of data and coming up with answers, Watson doesn't have another occupation.

FLATOW: Let's go to a quick question from Reginald in Cleveland. Hi.

REGINALD (Caller): Hello.

FLATOW: Hi, there. Quick.

REGINALD: I wanted to ask, like, had they seen any progression or - I don't know the exact word - maybe like growth - from when Watson was created up until, like, now, like, did it...

FLATOW: Yeah. Does it learn anything?

Mr. BAKER: Oh. When Watson started out four years ago - and my book really tells the story of the growth of this machine from really just a pile of software into an aspiring champion.

In the beginning, it was clueless. It didn't - it really had trouble understanding complex English, and it came up with ludicrous answers. Although I have to say, it still does occasionally.

FLATOW: Yeah. I noticed that in playing online. So what are the possible applications that we might make?

Mr. BAKER: Well, one of the things that IBM likes to talk about - you know, your previous guest was talking about this avalanche of data that we all have to deal with...

FLATOW: Right.

Mr. BAKER: ...and there's all kinds of valuable answers in that data, but none of us has the capacity to read it all. So if you're a neuroscientist, for example, 50,000 academic papers and studies were published last year on neuroscience. You can't read them all, but Watson can. And so if you have, let's say, a patient who comes into a hospital with symptoms - it's like a Dr. House scenario - Watson can read all of the literature that was published in the last year and look for correlations and come up with hypotheses about what that person might have. And some of them will be off the wall, but the humans can say, well, that one doesn't make sense, and that one doesn't make sense. But actually, that one - we hadn't thought of that. That's an idea.

FLATOW: Yeah. Because we couldn't search it all.

Mr. BAKER: We couldn't search it all.

FLATOW: And it's got enough intelligence to know how to weed through...

Mr. BAKER: It...

FLATOW: ...and find out what you're looking for.

Mr. BAKER: It has enough intelligence to make a good try, anyway.

FLATOW: Yeah.

Mr. BAKER: Come up with a stab.

FLATOW: And this could be a product IBM puts out.

Mr. BAKER: Oh, they have big hopes for it...

(Soundbite of laughter)

Mr. BAKER: ...in finance and in medicine, in pharmaceutical research. They have high hopes for this machine.

FLATOW: Well, we'll be watching next Monday through Wednesday. Cancel -cancel your Valentine's plans or make them more intimate...

Mr. BAKER: That's right.

FLATOW: ...for a Monday.

Mr. BAKER: Have a party.

FLATOW: There you go. Have a "Jeopardy" party and watch Watson take on Ken Jennings and...

Mr. BAKER: And Brad Rutter.

FLATOW: And Brad Rutter. The two top heavyweights in "Jeopardy".

Thank you, Stephen.

Mr. BAKER: It's been my pleasure, Ira.

FLATOW: Stephen Baker, former technology writer for Business Week, author of the new book "Final Jeopardy: Man vs. Machine and the Quest to Know Everything." It's available in eBook form now. You can download the final chapter next week after the episode airs.

So we're going to take a break and switch gears. Speaking of Valentine's Day, we're going to come back and talk about the science of kissing, yes, the science of kissing, with SCIENCE FRIDAY. Stay with us. We'll be right back.

Copyright © 2011 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.