Human Or Machine: Can You Tell Who Wrote These Poems?
ROBERT SIEGEL, HOST:
Computers have been inserting themselves into activities long reserved for humans. They can play on a par with the best chess player in the world, the best poker players. They've even done well against humans in the game show "Jeopardy!" But can a computer write a short story or a sonnet that's indistinguishable from what a human could produce?
Well, that's the question we take on for this week's All Tech Considered.
(SOUNDBITE OF MUSIC)
SIEGEL: NPR's Joe Palca is here to walk us through this. He first told us about a competition sponsored by computer scientists at Dartmouth College that tried to answer that question last year for his series, "Joe's Big Idea." Joe, thanks for coming in.
JOE PALCA, BYLINE: You're very welcome.
SIEGEL: Why did the Dartmouth scientists want to get people to create computer programs that can write sonnets or short stories in the first place?
PALCA: Well, the idea was to see if they could understand human nature enough and computer science enough to pass something that's called the Turing Test, named after the British computer scientist Alan Turing, who said, if you can pass the Turing Test, it means you've written a program that generates an output - a sonnet, in this case - that's indistinguishable from what a human sonnet writer could do.
SIEGEL: Well, I was actually a judge in this competition.
SIEGEL: I got 10 poems, and I had to pick out which were written by computer programs and which were written by human poets.
PALCA: Right. And now what we're going to do, Robert, is - we have two more poems for you to listen to and judge. So let's hear the first one now.
UNIDENTIFIED WOMAN: Create an endless voting paradox until another day of constant leaving becomes a tiny little music box and our ideas have become misleading. The people know exactly where or whether. Remember me without an honest word. The very moment we belong together - another simple story so absurd.
I take a message from a repetition and never ever follow my regards or not forget the secret definition or come together like a house of cards. And I consider you a friend of mine. Think about the other equals sign.
PALCA: OK that was the first one - 14 lines.
SIEGEL: Sonnet - the first, yeah, right, right.
PALCA: OK, and now let's hear the second.
UNIDENTIFIED WOMAN: Adjust my aperture towards the deep sky. Behold the gems of heaven on this world. Pray my gaze will not be sullied by the light from the strip and the street malls and suburbs. Discover in the glass the Pleiades, those sisters. Seven sapphires in the dark glow pretty, keen with visibility, the celestial beings who guide my heart.
But through the other micro lens I see an eye for science, an iris clear-blue, more stirring than the Whirlpool Galaxy, much brighter than the craters of the moon. Which lens could I look through for a longer time, observing each vast creation sublime.
PALCA: OK, so was the first one human or computer, do you think?
SIEGEL: The first one was a computer, I think.
PALCA: And the second, that would be human, well?
SIEGEL: And the second one was human, yes.
PALCA: Well, actually, you're correct. The computer-generated poem came from a team that submitted to this Turing Test competition from the University of Southern California Information Sciences Institute, and the other poem was written by a human being (laughter) - Caroline Weinroth, who responded to ALL THINGS CONSIDERED's Twitter call out for sonnets. So thanks, Caroline.
But we're really interested in the computer program. That's the one we want to discuss, and I want to bring in computer science professor Daniel Rockmore, the man behind this big Dartmouth competition, to tell us more about it.
SIEGEL: Professor Rockmore, you know, the computer-generated poem that we just had recited here, to me - create an endless voting paradox until another day of constant leaving becomes a tiny little music box - it could have become a quarter pound of lox. It makes no less sense.
SIEGEL: The computer-generated poems have rhyme. They have meter, but they don't mean anything. And the other poem is an image, and it's about...
DANIEL ROCKMORE: Yes.
SIEGEL: ...Observation and light, and it's - I mean, to me, it's a pretty unbridgeable gap there. What do you think?
ROCKMORE: Well, so first of all, I agree with what you said. On the other hand, having heard it declaimed so beautifully, I wasn't fooled, but I enjoyed it.
ROCKMORE: I mean, I think what you're referring to or how I view it is, you know, narrative is a very difficult thing, I think, to program at this point. You can get sensible phrases, but then stringing them together to construct an overall narrative image is still quite a challenge, and that was really the point of the competition. And I hope we succeeded a little bit in that.
SIEGEL: Well, you know, from the 10 poems that I judged in the earlier competition, I found there were some by humans that were such well developed or insightful images that the thought that a computer could do that - I would be both amazed if it could. I'd be terrified if it could, actually, because it would suggest a knowledge of human experience (laughter), and I don't think machines have or should have, frankly. But you think it's possible?
ROCKMORE: I mean, you phrase it in an interesting way. I mean, I don't know that I'd be terrified by that thought. I certainly would be impressed. Storytelling, I think many people have said, is maybe one of the things that is most human. And maybe in this competition and by the ease with which you determined the machine sonnets, maybe you're just, you know, reinforcing that. Maybe the competition is reinforcing that.
PALCA: The thing that intrigued me about the competition to begin with is, you have to understand humans before you can even start to think about writing a computer program. And so that's why I thought you had to understand something very fundamental about the creative process before you could write a computer program that could emulate it in a way that humans would be fooled by.
ROCKMORE: I think that's true. And what the winning program did, actually, was try to think hard about the creative process of a sonnet - how you might open a sonnet, how a human being might rhyme or almost rhyme words. So there were - it's certainly edging toward that.
And I think that what happens once you start to dig deep into the particular process of sonnet writing, for example, is that you discover just how complicated it is. And so, like, with many projects where you're thinking about people and what makes us human, you do begin to discover the complexity of that, and you appreciate it even more.
SIEGEL: Yeah. I was impressed with the almost-rhyme capacity of the programs. The big difference I found was that the human poets could hang with an idea for 14 lines, whereas I felt that the computer just couldn't hang in there (laughter). It would be in...
ROCKMORE: Yeah, well...
SIEGEL: ...One place in the beginning, and then someplace else in the middle and someplace else yet again at the end.
ROCKMORE: Yeah. I think that when we do this competition next year, this team - the USC team - was very interested to hear that, and they're already working on their next version.
ROCKMORE: So we'll see if they get you next time.
SIEGEL: OK. Well, thanks to professor Daniel Rockmore of Dartmouth College and to NPR's Joe Palca for this. You can test whether you can tell the poetry of humans from that of machines. We posted some poems on our All Tech Considered blog, which you can find at npr.org/alltech. Thanks, guys.
PALCA: You're welcome.
ROCKMORE: Thank you.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.