The Inner World of Ripley the Robot Plenty of computers know thousands of words and complex rules of grammar. But they're dismal communicators. So a group of scientists at MIT is trying a different approach: They're creating robots that learn what words mean the same way humans do.

The Inner World of Ripley the Robot

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


This is MORNING EDITION from NPR News. I'm Renee Montagne.


In the final part of our series on language, NPR's Jon Hamilton visits a small boy named Liam(ph), and a robot named Ripley, who are both learning to talk.

JON HAMILTON: The first word from a child's mouth may be mama or dada. Pretty soon, though, he's talking about really important stuff.


BETSY LAVIN: That's right. That's a truck.

HAMILTON: Liam Gregory monitors trucks from the second floor of his home in northern Virginia. His mom is Betsy Lavin. She says Liam has just started talking.

LAVIN: He is 14 months old, and he is saying probably about six or seven words consistently, along with a couple phrases.

HAMILTON: One Liam's favorite words - when trucks aren't around - is ball, like the bright yellow softball that's just rolled under a table.

LAVIN: Do you want to find the ball? Where's the ball? Where did we leave the ball? Can you find it? Yeah! There it is!


HAMILTON: Deb Roy, a researcher at MIT, says that's not nearly enough.

DEB ROY: Language is inherently a social activity. It's a symbolic activity. It's far ore than just vocabulary and grammar. That's just - that's the surface stuff.

HAMILTON: Kai-Yuh Hsiao is a graduate student who has spent what is probably an unhealthy amount of time with Ripley. When they're together, they share a physical world, just like Liam and his mom. This one isn't very big or complicated, but it's one they both can touch and see and hear.

KAI: Ripley lives in a tabletop domain. It's got this white tablecloth here. We've got objects on the table and it's got this one camera. And so right now, it's looking at this blue beanbag and this red ball.

HAMILTON: Ripley constantly updates the robotic equivalent of a mental image of the objects. The robot knows what they look like and where they are, and it can answer questions about them.

HSIAO: Where is the blue one?

RIPLEY: At the center.

HAMILTON: Ripley learns about this tabletop world the same way Liam learns about his house: by observing and putting anything within reach in its mouth.

HSIAO: Hand me the blue one.


HAMILTON: Deb Roy says Ripley doesn't know a lot of words, but it really does know what it's talking about.

ROY: This is not a language-processing machine that's just pushing around symbols. You know, I put this thing down in front of Ripley, suddenly we have something to talk about - we, Ripley and I - and we're talking about that thing out there in the world that we both can touch and see and feel.

HAMILTON: Roy says that's the starting point for real communication, the first layer of iceberg beneath the surface. To go even deeper, Roy says computers need to figure out not only the physical world, but the minds of the people who live in it.

ROY: Ground level is the physical level, okay. Social - how do I know what's going on in your head right now? Why are you saying the things you are? What are the things you are likely to believe at this moment and so forth. These are all inferences. These are all leaps of faith, guesses I'm making, right, about what's going on in your head.

HAMILTON: A bit like reading a person's mind. Scientists call this theory of mind or theory of other minds. This ability represents a big part of the mental iceberg that sits beneath everything we say. Moms are especially good at reading minds.


HAMILTON: Like when Liam uses his yellow ball for a gravity experiment.

LAVIN: It's okay. It's heavy, isn't it?


LAVIN: Yeah, you're okay.

HAMILTON: Lavin says her son is still working on his own ability to empathize. It's something most kids start to do around age two, but it take years to develop fully.

LAVIN: I don't know if he would understand a facial expression that portrayed hurt or sadness or something. If I'm changing him, and he kicks me and I'll say, no, that hurts Mommy, he doesn't understand the concept of hurt and he'll think it's funny and he'll keep kicking.

HAMILTON: That will change by the time Liam is riding a bike, but how? Deb Roy at MIT says even something as complex as empathy appears to have its roots in the physical world. Ripley, the robot, can't really empathize, of course, but Roy says it can see its tabletop world from someone else's point of view. Hsiao shows how a conversation can change the computer image that shows Ripley's view of the world.

HSIAO: Now, you can see what it did was exactly what it did last time. It reached down and handed me the blue beanbag, because the blue beanbag was, in fact, on my left. But if you look at what the screen has on it now, the blue one is actually on my left from my perspective, so it's actually imagining its world from my eyes.

HAMILTON: A fairly simple trick for a robotic brain, but Roy says it's also a baby step toward a robotic theory of other minds and even empathy. The robot can see the physical world the way it appears to someone else and that adds another layer of meaning to the words it hears and speaks.

ROY: If the words are the tip and the iceberg is this very rich mental structure that is being evoked, both in your mind and mine, to the degree to which they somehow overlap to the degree to which what's in your head and my head is aligned, we're successfully communicating.

HAMILTON: Ripley aligns its head with a person's only when they're both talking about objects on the tabletop. But people have no such limits. Our conversations draw on a huge store of common experiences and emotions and that's why just a few words between a 14-month-old child and his mother can convey an iceberg of meaning.

LAVIN: You calling Daddy?

GREGORY: Dadd-u, Dadd-u.


HAMILTON: Liam has trouble with the whole dialing thing, so his mom helps him out.

LAVIN: Hello. Is it Daddy? Is it for Liam? Is it for you?


LAVIN: Is it Daddy? Here you go.

HAMILTON: Jon Hamilton, NPR News.


INSKEEP: You can watch a video of Ripley exploring his tabletop world, and also get the answer to the related question, Does my dog really get me? by going to

Copyright © 2006 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.