Monkeys Learn to Control Robotic Arm With Brainwaves

May 29, 2008 at 6:45 PM EDT

Loading the player...

Advances in brain-controlled prosthetics reached new heights as researchers at the University of Pittsburgh announced the successful use of a prosthetic arm linked directly to the brain of a monkey. Lead researcher Andrew Schwartz discusses the findings.

TRANSCRIPT

JEFFREY BROWN: It’s a remarkable feat of monkey and machine. Researchers reported today that a pair of monkeys learned how to control a robotic arm and feed themselves by using only their brainpower.

Scientists in Pittsburgh reported in the journal Nature that they implanted tiny electrodes in the brains of this monkey and one other.

As described in their paper, the electrodes were implanted in a part of the brain which controls movements. Signals from the brain were wired directly to a computer, which was attached to the robotic arm.

The monkeys were able to control the robotic arm to bring them food, even as their own arms were restrained.

By operating directly on the brain, this monkey-think-monkey-do experiment goes further than earlier efforts to create similar technologies for human patients with spinal cord and other injuries.

Other avenues of research that are not connected with brain surgery have already been tried with humans and new prosthetics. All are imperfect and in the experimental stages.

Now, scientists are looking at today’s report to see whether they, too, can work directly on pathways to the brain.

Teaching monkeys to use robots

JEFFREY BROWN: And for more on this finding, we're joined by the study's senior author. Andrew Schwartz is professor of neurobiology at the University of Pittsburgh School of Medicine.

Well, let's step back, if you would, and explain a bit more about how this is done. Electrodes are implanted in a part of the brain that controls movement, and this is connected to a computer? Walk us through this a little bit.

ANDREW SCHWARTZ, University of Pittsburgh: Right, so it really has three components. The first are the microelectrodes. These are small electrodes about the size of a human hair. These pick up single impulses from cells in the brain. And we pick up enough of those that we can extract a signal that's processed by a computer.

So these electrodes send their signals to a computer. The computer processes these signals and generates what we consider the monkey's intention to move. And that intention is then processed, once again, into a language that the robot can understand, and the robot then moves along with the animal's desire.

JEFFREY BROWN: So do you have to train the monkeys to use the mechanical arm? And how long does that take?

ANDREW SCHWARTZ: Well, there's actually two phases to the training. First, we have to familiarize the monkey with the whole idea that there's an arm that it can control. And we do that with a joystick.

So before we implant any electrodes, we just train the monkey to move the arm with a joystick, so it learns about the arm. And then, after it's familiar with that, we implant the electrodes and we train the monkey in stages.

What we do is, first, train the monkey to move the arm toward the target. Once it's at the target, we have to have it stabilize and then close the gripper and then bring it back. So there's really four stages to that.

JEFFREY BROWN: But in essence, in the end, the monkey is thinking the movement? Is that a way to say it?

ANDREW SCHWARTZ: Right. So we believe it'd be analogous to moving your own arm. And one way to think about it is as if you were moving a mouse and generating movement on a computer screen with a cursor, so you move that mouse with your hand and you see the cursor move. And pretty soon you stop even thinking about your hand moving at all with the mouse and just look at the cursor.

JEFFREY BROWN: And how much were the monkeys able to do? And what are the limits to their movement?

ANDREW SCHWARTZ: Well, right now, they're able to reach out in space, grasp a small piece of food, and bring it back to their mouth. And it's a pretty skilled, accurate movement. We think it takes about two millimeters of accuracy in order to grasp the food, and they do pretty well.

So they can move the arm anywhere in space in front of them and grasp the food. Now, what this device is missing is a wrist and hand and fingers. So it's actually pretty difficult to orient the gripper to pick up the food.

One step closer to human patients

JEFFREY BROWN: How do you define the important, the key advance here, beyond what researchers have been able to do so far?

ANDREW SCHWARTZ: Well, really, this is a step in a process that has been going on for a number of years now. And what this signifies is this is really the first time we've been able to use this kind of brain control with a physical device.

Up to this point, the work has been almost exclusively done with displays that are on the computer screen. So this is the first time that the animal sat with the robot and worked with it directly.

JEFFREY BROWN: So now the question is, how far off are you from applying this kind of work to people and for what kind of injuries?

ANDREW SCHWARTZ: Well, there's already been preliminary work where a number of patrons have been implanted with exactly the same kind of electrodes that we're using here in the monkeys.

We think that, in the next two years, we'll be able to have human patients, a few human patients for research purposes, controlling the same type of device and probably doing much better than what these monkeys were able to do.

JEFFREY BROWN: And the idea would be to manipulate the prosthetics?

ANDREW SCHWARTZ: Right. So this would be, perhaps for amputees or for spinal cord-injured patients or for patients with other types of paralysis.

JEFFREY BROWN: What are the key hurdles before you can get to that?

ANDREW SCHWARTZ: Well, there are two things. If we want to add a wrist and fingers, we need to understand some more about how the brain encodes these movements.

And then the other thing, there needs to be a number of technical improvements. So right now, the electrodes, after a while, they've been implanted permanently in the brain, they become encapsulated with scar tissue and our signal degrades.

The other thing we would like to do is to miniaturize the whole set of pieces of equipment from the connectors. And we'd like to use transmitters to wirelessly send these signals out of the brain. The computers can be miniaturized to a single chip, and then even the prosthetic arm itself can be made much nicer.

Decoding the brain's language

JEFFREY BROWN: You just raised something that I want to ask you more about. You raised how much we know about how the brain encodes movement. How much do we know about that?

ANDREW SCHWARTZ: Well, so this is actually a demonstration showing about how much we know at this point. So because these signals are relatively straightforward for us to interpret, we can get this kind of movement.

There are many other issues about brain activity that we don't understand, but this technology, because we can record many cells at the same time, are allowing us to consider the brain differently than we did before. We can now consider the brain as a complex system, with many elements working simultaneously, and we can begin to understand how these neurons speak to each other.

JEFFREY BROWN: I do want to ask you before we go about the monkeys themselves, because I think a lot of people will probably wonder, "Why use monkeys?" Are they at any risk or harm in the kind of work you're doing?

ANDREW SCHWARTZ: Well, these monkeys are actually quite comfortable. We just restrain their arms by their sides to encourage them to use this device.

And once again, the electrodes we're implanting are exactly the same type that are implanted in humans. And there are no -- I really don't think there's any discomfort. The monkeys have to be comfortable in order to work.

JEFFREY BROWN: And what happens next at your lab? What are you working on next with them?

ANDREW SCHWARTZ: Well, we're working really hard on trying to understand how the brain encodes finger movements and wrist movement. And then we aim to make a prosthetic device that has a very natural hand and wrist. And actually, from an engineering point of view, this is really complex, because the hand as a machine is much more complex than the shoulder and elbow.

JEFFREY BROWN: All right, Andrew Schwartz at the University of Pittsburgh, thank you very much.

PBS NewsHour allows open commenting for all registered users, and encourages discussion amongst you, our audience. However, if a commenter violates our terms of use or abuses the commenting forum, their comment may go into moderation or be removed entirely. We reserve the right to remove posts that do not follow these basic guidelines: comments must be relevant to the topic of the post; may not include profanity, personal attacks or hate speech; may not promote a business or raise money; may not be spam. Anything you post should be your own work. The PBS NewsHour reserves the right to read on the air and/or publish on its website or in any medium now known or unknown the comments or emails that we receive. By submitting comments, you agree to the PBS Terms of Use and Privacy Policy, which include more details.