NPR logo

Monkeys Control Robotic Arm Just by Thinking

  • Download
  • <iframe src="https://www.npr.org/player/embed/90974686/90974786" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Monkeys Control Robotic Arm Just by Thinking

Research News

Monkeys Control Robotic Arm Just by Thinking

  • Download
  • <iframe src="https://www.npr.org/player/embed/90974686/90974786" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

RICHARD HARRIS, host:

Up next, the latest in the search for the perfect prosthetic. Imagine eating dinner with your hands tied behind your back, not easy to do. But it gives you an idea of the everyday challenges that some paralyzed people face. They have no way of helping themselves to another piece of bread, or another shake of salt. But new technology could help those tasks easier. Neurobiologists at the University of Pittsburgh recently gave monkeys the ability to control a robotic arm, just by thinking about it.

They inserted tiny hair-like electrodes into the monkey's brain that picked up electrical nerve signals from the motor cortex. The monkeys learned that certain thoughts could cause the robot arms to move, and eventually they will - were able to use the mechanical arm to pick up fruit, and I guess marshmallows and eat it. Joining me now to talk about the prospects of this technology is Andrew Schwartz. He's a professor of neurobiology at the University of Pittsburgh, and he joins us today in our studio in New York. Welcome to the program, Dr. Schwartz.

Dr. ANDREW SCHWARTZ (Neurobiology, University of Pittsburgh): Hello, Richard.

HARRIS: If you'd like to join the discussion give us a call. Our number is 1-800-989-8255. That's 1-800-989-TALK. If you want more information about what we'll be talking about this hour, you can also go to our website at www.sciencefriday.com. We also have video there of a monkey feeding itself using the robotic arm. So tell me about this. Where did you get the idea that you could sort of tap in to the electrical signals of - in the brain, and actually get it to command machineries? Sounds almost science fiction.

Dr. SCHWARTZ: Well, this is actually the culmination of work that's been going on for more than 25 years. What we are doing quite awhile ago is recording individual neurons and then - so neurons are the cells of the brain, and we record activity, the electrical activity as each one fires, and what we found is that as you move your arm in different direction, these neurons seemed to be quite sensitive to that. So each neuron has the, what we call a preferred direction, that likes to fire in a direction of arm movement.

HARRIS: Right.

Dr. SCHWARTZ: And then once we know which direction each cell likes to fire in, we can add responses from many of these cells together, and get a very accurate representation of what the arm movement is.

HARRIS: Hm. So how many electrodes you have to do, to tap into these, to do the signals.

Dr. SCHWARTZ: Well, to get a really high density arm trajectory of the path of the arm, it takes about 30 to 100 electrodes recording about that many neurons.

HARRIS: Well, that sounds like a lot of detailed work to get this set up.

Dr. SCHWARTZ: Well, actually, that's why it's taken about 20 years, so that we are - could develop this technology to allow us to record all these cells at the same time.

HARRIS: Right. And I guess you also need the cooperation of the monkeys. At what role do they play?

Dr. SCHWARTZ: Well, initially we had the monkeys working at what we call, open loop conditions, and they didn't even really know that we're recording the nodes from their neurons. And one of the big advances was to close the loop, to give them an indication of what the signal was that we are recording from their brain. And once they do that, they could actually learn to use this device and improve the way their neurons fired.

HARRIS: So, how - walk us through this experiment and how do you train them to associate their thoughts with the movement of an arm. It's seems - that's hard.

Dr. SCHWARTZ: Well, the first thing we had to do was to familiarize them with the whole concept of controlling the tool. In this case was a robot arm, and what we had them do is move the joystick back and forth, and that joystick was linked to the movement of the arm, and they learn just by moving the joystick that they could bring food to their mouth.

So, that gave them the general idea what we wanted them to do. And then what we did, is we implanted these electrodes in their brain and had them basically think about moving the arm, and what we did is we held a piece of food up in front of them, and then we moved the arm for them, and as we did that, we saw that neurons in their brain fired as if they were imagining that they're actually moving that.

And once we saw the neurons fire, we knew to recognize those and use those in our control scheme. And then we just proceeded step by step. So, first we had the animal move the arm to the vicinity of the food, and then stabilize the arm, and then learn to close the grip around it, and finally to bring the arm back to the mouth.

HARRIS: Wow. And they were rewarded with...

Dr. SCHWARTZ: Piece of food, sometimes a zucchini, apples, oranges, marshmallows.

HARRIS: Ah. We'll be right back after this short break.

(Soundbite of music)

HARRIS: From NPR News, this is Talk of The Nation: Science Friday and I'm Richard Harris. We're talking this hour about monkeys controlling a mechanical arm just by thinking about it. My guest is Andrew Schwartz, a professor of neurobiology at the University of Pittsburgh. And you were saying that you took a lot of steps to get this going. How long did it take you to train the monkeys to actually make the arm work?

Dr. SCHWARTZ: Well, that's sort of a tough question to answer, because as we are training the monkey, we're also doing a lot of debugging of both our equipment and software. So, that ended up taking a few weeks. But I think that if we're to do it again, right away now that we have everything fixed, we could probably do it in two to three days.

HARRIS: Really? That they could learn - they could go through and learn the whole process that quickly.

Dr. SCHWARTZ: Well, they could learn the rudimentary aspects of the task. Of course, the more they practice, the better they get.

HARRIS: Right. And how long did the electrodes last in the brain?

Dr. SCHWARTZ: Well, right now the - one of the monkeys that we implanted has been implanted for a year and a half, and we're still getting useful signals from the brain.

HARRIS: Mm. Is that - that's a record or, I mean, is that unusual to have it that long, or is that sort of typical?

Dr. SCHWARTZ: No, but sometimes we've had monkeys last as long as seven years with the electrodes implanted in their brain. The longevities of these electrodes have been increasing over the recent years.

HARRIS: I see. Because, I understand, early on that was a real limitation of the work.

Dr. SCHWARTZ: Well, still a limitation in that scar tissue tends to form around these tiny electrodes, and then our signal degrades. And actually the signal does get a little worse, but we can still use some aspects of it.

HARRIS: Mm. Well, let's go to the phones here. Our number is 1-800-989-TALK. Please give us a call and join the conversation. Let's hear from Brian in Lewis, Delaware. Brian, welcome.

BRIAN (Caller): Hi. How you doing?

HARRIS: Good. How are you?

BRIAN: Very well. Thank you for taking my call. I'm just curious if this research is sort of going towards paralysis victims perhaps? Looking at perhaps through these signals and reconnecting to limbs - any sort of research towards that? And I'll take my answer off the air. Thank you very...

HARRIS: Thanks.

Dr. SCHWARTZ: Well, that's exactly what we're trying to show here. We want to help patients that can't move their arms, or that have amputation of the arm. So, what we're showing right with this research is that, you know, that we can do self feeding basically.

HARRIS: Mm. And there have been some experiments in human beings, right? Well, how do the monkey experiments compare?

Dr. SCHWARTZ: So, we're using the same exact electrodes that have been implanted in a number of humans. And those humans were able to control cursors in a computer monitor.

HARRIS: I see. So, what's the difference between that and the robotic arm?

Dr. SCHWARTZ: Well, there's a big difference between working in a computer and working a physical device. This physical device you know, doesn't work exactly the way we command it to. There's things like marshmallow sticking to the hand, or fruit slipping from the gripper, and actually the monkey has to learn to handle all these conditions.

HARRIS: Right. And when I reach out and touch something of course, I get feedback from my fingers, is - and that helps me manipulate whatever I'm holding. Do the monkeys get any kind of feedback from that?

Dr. SCHWARTZ: Well, right now, the only feedback the monkeys get are - is from their vision. As we increase the complexity of the task, and start using fingers in a dexterous way, we fully realize that we're going to need to have tactile sensation. And what we hope to do is use sensors in the prosthetic fingers, and then take that information and feed it back to the brain, stimulate sensory regions.

HARRIS: Ah. And that's another wiring job. Is that more difficult?

Dr. SCHWARTZ: Well, it's going to be another challenge. But hopefully, if we start simple, we'll be successful.

HARRIS: OK. Let's take another call. Let's hear from Arthur in Binghamton, New York. Welcome to Science Friday.

ARTHUR (Caller): All right. Thank you very much for taking my call. I've very enjoyed hearing about the monkey robotic successes that you've been making, and I just a bit - wanted to get a bit of a monkey off my own back. And that's about a very serious threat to the northeast watershed, involving the fracturing of gas, using the exemption of the Clean Water Act and the Safe Drinking Water Act, which could potentially threaten everything. And wouldn't we be better off putting a robotic monkey's arm down there, instead of trying to mock around with threatening the greater watershed like that?

HARRIS: OK. Well, thanks for your call. It's a little off-topic, but I appreciate you're calling in. Bye-bye. So, tell me about how many other people, or how many of the labs are doing this kind of research around the world.

Dr. SCHWARTZ: Well, there are a number of other labs doing this similar approach, but what's really exciting is the whole field of what we call neural-engineering, that's just exploding now. This whole idea of linking engineering with neuro - basic neurosciences are really hot topic, and the community is growing explosively.

HARRIS: And obviously, you have some very practical end points here. But what are you learning about the brain in the process of doing this?

Dr. SCHWARTZ: Well, so the technology we've developed allows us to record from a group of neurons simultaneously, and that's much more representative of how the brain really functions. Every time we do anything, there's millions and millions of neurons that change your activity, and the better we can sample that network activity, the better we can understand what's going on. And we're just beginning - very early stages to try to understand the brain from this perspective and this prosthetic success is an example of that.

HARRIS: Any prospect of doing this with essentially remote sensing of the brain, not having to put electrodes into it?

Dr. SCHWARTZ: Well, so a number of groups have been trying to do this from outside the brain with EEG, and there has been some limited success. But I think the kind of details we really need to get full prosthetic control, is going to require this kind of technology.

HARRIS: OK. Let's take another call. Let's hear from Ryan in Madison, Wisconsin. Ryan, welcome to Talk of The Nation: Science Friday.

RYAN (Caller): Yeah, hi. Really the reason I'm calling - I wasn't exactly forthright with your screener. My call wouldn't have been able to go through. I mean, I know I'm not the only one that's concerned about robots taking over the world and now, there's going to an unholy alliance with the monkeys controlling them? This is how they get back to us for doing what we've done to the rain forest. I'll take my comments off the air.

(Soundbite of laugher)

HARRIS: OK.

RYAN: I just don't think that's a great idea. Monkeys controlling robots with their thoughts, come on.

HARRIS: Well, I think that there's a deeper purpose, is there not, for this?

Dr. SCHWARTZ: Well, right. So, we're using monkeys, because they have very human-like arms and they can participate in these kind of natural reaching tasks.

HARRIS: Yeah. Is it painful to the monkeys to have these implants?

Dr. SCHWARTZ: Not really. Not at all. In fact, these implants are exactly what we use in humans and the humans are comfortable with these.

HARRIS: Good. OK. Yeah. It's a - I guess it's a good sign if you do the same experiments on people, that you do on the animals. It's, yes. Let's take another call. This is John in Portland, Oregon.

JOHN (Caller): Hi. How are you?

HARRIS: Good. Welcome to Science Friday.

JOHN: Thanks. There's a lot of talking and articles written about this, that talk about how this could potentially benefit humans. My question is more about the monkeys, you know, do monkeys have a conscience? Is there anything we can learn about what's going on inside of the monkey's brain, and possibly other animals and you know, what could that potentially lead to?

Dr. SCHWARTZ: Well, that's a good question. As we study movement, and as it becomes more and more complex, we realize that a lot of these relational movements are really the output of our intentions. And if you link intentions a little bit more, you start to get things like consciousness, decision making, intention to move, and we are working toward that. Of course, it's pretty hard to assess consciousness in a monkey, because we can't speak to them. But as these experiments progress to humans, I think we can really begin to attack some of these issues.

HARRIS: Mm. So, yeah, I guess that would be interesting, if you could actually get in and study that. But maybe not electrodes. This probably may not be the way to get into that sort of topic. So - so...

Dr. SCHWARTZ: So, actually we are really interested in learning how these network properties produce things like learning, and thinking, and decision making. So, we're actually working real hard to understand how groups of neurons interact to produce this.

HARRIS: Yeah. And along those same lines, do you have any idea whether the monkeys perceive that the arm is moving is their own arm, or do they just sort of see it as a video game? Any way you can suss that out?

Dr. SCHWARTZ: Well, we can't ask them. But what we see is that the monkeys seem to embody this device, and that they do seem to consider it as part of their own body or at least are very comfortable with it, because they do things like, lick the hand when it has marshmallow residue, and they use the hand to shove food and rearrange it around inside their mouth, and they can stop and control it very easily.

HARRIS: Mm. And what happens to their comparable arms when they're manipulating this robotic arm?

Dr. SCHWARTZ: Well, we just have them gently restrained and some tubes along the side of their arms. And although the monkey may squirm a little bit to get comfortable as he's doing the task, most of the time he's pretty quiet and doesn't really try to move his arms.

HARRIS: So, it's not the same exact electrical signals that they'd be using to try to move their actual arms.

Dr. SCHWARTZ: No, it's pretty clear that they can disassociate driving their own arms from driving the robot arm.

HARRIS: Ah, very good. Let's take one more call. This is Jason in oops - yeah, Jason in Portland, Oregon. Jason, welcome to Science Friday.

JASON (Caller): Hi. Thanks for taking my call. My question, related to research that was done by Dr. Piatauski (ph) back in the late '80s. I remember him using electronics to control human muscles. And I was wondering if this had been thought of as a way to actually control muscles) instead of robotic prosthesis.

Dr. SCHWARTZ: Well, exactly. Someday we'd like to be able to take this (unintelligible), and for instance patients that are paralyzed, we'd like to re-activate their own muscles, instead of having them use (unintelligible) prosthetic device. That turns out to be a bit (unintelligible) of a control problem, but a lot of people are working on that right now.

HARRIS: Ah, that's fascinating. That would indeed be great. Well, thank you very much for coming in. We've run out of time, but it's a really interesting conversation and as things progress, perhaps you can come back and give us an update.

Dr. SCHWARTZ: Thanks for having me, Richard.

HARRIS: Andrew Schwartz is professor of neurobiology at the University of Pittsburgh and again, thanks for joining us.

Copyright © 2008 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.

Comments

 

Please keep your community civil. All comments must follow the NPR.org Community rules and terms of use, and will be moderated prior to posting. NPR reserves the right to use the comments we receive, in whole or in part, and to use the commenter's name and location, in any medium. See also the Terms of Use, Privacy Policy and Community FAQ.