The Inner World of Ripley the Robot Plenty of computers know thousands of words and complex rules of grammar. But they're dismal communicators. So a group of scientists at MIT is trying a different approach: They're creating robots that learn what words mean the same way humans do.
NPR logo

The Inner World of Ripley the Robot

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
The Inner World of Ripley the Robot

The Inner World of Ripley the Robot

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


This is MORNING EDITION from NPR News. I'm Renee Montagne.


And I'm Steve Inskeep. Good morning.

Even a child can say a lot with just a few words, especially if one of those words is no! But computers are still pretty clumsy with language. Think about your last phone call to a computerized reservation system.

The problem is that words are really just the tip of a linguistic iceberg, and researchers are just trying to figure out what makes up the rest of that iceberg.

In the final part of our series on language, NPR's John Hamilton visits a small boy named Liam(ph), and a robot named Ripley, who are both learning to talk.

JOHN HAMILTON reporting:

The first word from a child's mouth may be mama or dada. Pretty soon, though, he's talking about really important stuff.

LIAM GREGORY(ph): Truck.

Ms. BETSY LAVIN(ph) (Mother of Liam Gregory): That's right. That's a truck.

HAMILTON: Liam Gregory monitors trucks from the second floor of his home in northern Virginia. His mom is Betsy Lavin. She says Liam has just started talking.

Ms. LAVIN: He is 14 months old, and he is saying probably about six or seven words consistently, along with a couple phrases.

HAMILTON: One Liam's favorite words - when trucks aren't around - is ball, like the bright yellow softball that's just rolled under a table.

Ms. LAVIN: Do you want to find the ball? Where's the ball? Where did we leave the ball? Can you find it? Yeah! There it is!

(Soundbite of excited shriek)

HAMILTON: One reason Lavin and her son communicate so well is that they share a physical world. It's filled with stuff they both understand: trucks, cats, balls, books, telephones, and daddy.

But computers generally live in cyberspace, not our space, so they don't really understand much about balls, or trucks, or people trying to book a plane reservation. All they have is a dictionary full of words and some rules for stringing them together.

Deb Roy, a researcher at MIT, says that's not nearly enough.

Professor DEB ROY (Director, Cognitive Machines Group, Massachusetts Institute of Technology): Language is inherently a social activity. It's a symbolic activity. It's far ore than just vocabulary and grammar. That's just - that's the surface stuff.

HAMILTON: Roy is part of a team trying to create robots that can really talk. This kind of research, along with work on animal communication and human language disorders, is starting to reveal what lies beneath the surface stuff and it's changing the way scientists think about the nature of language.

Here's Ripley. It looks like a pint-sized version of that monster from the Alien movies. Picture the front half of a mechanical snake with a big metal mouth. Ripley also has a camera lens for an eye, a microphone for an ear, and a brain programmed in Linux.

Kai-Yuh Hsiao is a graduate student who has spent what is probably an unhealthy amount of time with Ripley. When they're together, they share a physical world, just like Liam and his mom. This one isn't very big or complicated, but it's one they both can touch and see and hear.

Mr. KAI-YUH HSIAO (Graduate Student, Massachusetts Institute of Technology): Ripley lives in a tabletop domain. It's got this white tablecloth here. We've got objects on the table and it's got this one camera. And so right now, it's looking at this blue beanbag and this red ball.

HAMILTON: Ripley constantly updates the robotic equivalent of a mental image of the objects. The robot knows what they look like and where they are, and it can answer questions about them.

Mr. HSIAO: Where is the blue one?

RIPLEY: At the center.

HAMILTON: Ripley learns about this tabletop world the same way Liam learns about his house: by observing and putting anything within reach in its mouth.

Mr. HSIAO: Hand me the blue one.


HAMILTON: By holding objects in its jaws, Ripley figures out how heavy they are and whether they're soft or hard. All this real-world information gets stored in the robot's computer brain.

Deb Roy says Ripley doesn't know a lot of words, but it really does know what it's talking about.

Prof. ROY: This is not a language-processing machine that's just pushing around symbols. You know, I put this thing down in front of Ripley, suddenly we have something to talk about - we, Ripley and I - and we're talking about that thing out there in the world that we both can touch and see and feel.

HAMILTON: Roy says that's the starting point for real communication, the first layer of iceberg beneath the surface. To go even deeper, Roy says computers need to figure out not only the physical world, but the minds of the people who live in it.

Prof. ROY: Ground level is the physical level, okay. Social - how do I know what's going on in your head right now? Why are you saying the things you are? What are the things you are likely to believe at this moment and so forth. These are all inferences. These are all leaps of faith, guesses I'm making, right, about what's going on in your head.

HAMILTON: A bit like reading a person's mind. Scientists call this theory of mind or theory of other minds. This ability represents a big part of the mental iceberg that sits beneath everything we say. Moms are especially good at reading minds.

(Soundbite of child crying)

HAMILTON: Like when Liam uses his yellow ball for a gravity experiment.

Ms. LAVIN(ph): It's okay. It's heavy, isn't it?

(Soundbite of child crying)

Ms. LAVIN: Yeah, you're okay.

HAMILTON: Lavin comforts her son with simple words: thumb, ball, heavy. That's the tip of the iceberg. What she's actually conveying is empathy and sympathy. I understand how you feel. I've felt that way, too. You'll feel better soon. Liam seems to understand, or maybe he's just distracted by one of the cats.

Lavin says her son is still working on his own ability to empathize. It's something most kids start to do around age two, but it take years to develop fully.

Ms. LAVIN: I don't know if he would understand a facial expression that portrayed hurt or sadness or something. If I'm changing him, and he kicks me and I'll say, no, that hurts Mommy, he doesn't understand the concept of hurt and he'll think it's funny and he'll keep kicking.

HAMILTON: That will change by the time Liam is riding a bike, but how? Deb Roy at MIT says even something as complex as empathy appears to have its roots in the physical world. Ripley, the robot, can't really empathize, of course, but Roy says it can see its tabletop world from someone else's point of view. Hsiao shows how a conversation can change the computer image that shows Ripley's view of the world.

Mr. HSIAO: Hand me the one on my left.

Now, you can see what it did was exactly what it did last time. It reached down and handed me the blue beanbag, because the blue beanbag was, in fact, on my left. But if you look at what the screen has on it now, the blue one is actually on my left from my perspective, so it's actually imagining its world from my eyes.

HAMILTON: A fairly simple trick for a robotic brain, but Roy says it's also a baby step toward a robotic theory of other minds and even empathy. The robot can see the physical world the way it appears to someone else and that adds another layer of meaning to the words it hears and speaks.

Prof. ROY: If the words are the tip and the iceberg is this very rich mental structure that is being evoked, both in your mind and mine, to the degree to which they somehow overlap to the degree to which what's in your head and my head is aligned, we're successfully communicating.

HAMILTON: Ripley aligns its head with a person's only when they're both talking about objects on the tabletop. But people have no such limits. Our conversations draw on a huge store of common experiences and emotions and that's why just a few words between a 14-month-old child and his mother can convey an iceberg of meaning.

Ms. LAVIN: You calling Daddy?

LIAM: Dadd-u, Dadd-u.

(Soundbite of phone being dialed)

HAMILTON: Liam has trouble with the whole dialing thing, so his mom helps him out.

Ms. LAVIN: Hello. Is it Daddy? Is it for Liam? Is it for you?

(Soundbite of laughter)

Ms. LAVIN: Is it Daddy? Here you go.

HAMILTON: Liam doesn't know he's supposed to talk into the phone, but he still conveys something a computer never could.

Jon Hamilton, NPR News.

(Soundbite of music)

INSKEEP: You can watch a video of Ripley exploring his tabletop world, and also get the answer to the related question, Does my dog really get me? by going to

Copyright © 2006 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.