Predicting the Future for Brain-controlled Devices

A paraplegic woman uses mind-controlled robotic arm to feed herself in a clinical trial at the University of Pittsburgh (photo via ExtremeTech)

When an outfielder catches a fly ball, he makes a series of calculations about where the ball is going to be so he can position himself to make the play. The best fielders can tell how fast the ball is going and its exact angle almost as soon as it comes off the bat.

They’re not working out the ball’s trajectory consciously, of course. They’re acting on instinct developed from years of practice at making predictions of where the ball will be, predictions based on watching a few seconds of the ball’s first few movements and what they know about the physics of how baseballs normally move.

Two researchers in the Department of Organismal Biology and Anatomy at the University of Chicago are using the same concept of predicting future movements to improve performance in a far more sophisticated scenario: a brain-machine interface that uses neural signals from the brain to move a device like a robotic arm. In a study published in the Journal of Neural Engineering, they showed how they were able to improve performance of such a system by adjusting the software algorithm that interprets the brain signals into motor commands to account for delay in the system and predict intended future movements.

There’s a delay from the time the brain thinks about moving to when the body actually begins to move—say, from when an outfielder thinks about sprinting to chase down a line drive to when he picks up his foot to start running. In a human, that delay is typically 100 milliseconds—one-tenth of one second. Our bodies are accustomed to the delay though, so we don’t notice it.

A system using a brain-machine interface to control a robotic limb adds more delay time, due to the time it takes to decode the signals and actually move the mechanical device. In research settings, as little as an additional 100 ms can start to impact the ability to perform tasks accurately.

“The brain is used to functioning on one time scale. It fires a certain amount of time before it expects to see movement,” said Aaron Suminski, a postdoctoral scholar who worked on the study. “What we’re trying to do with delay is take out the effects of the artificial system, and try to bring it as close to the real physiological system as we can.”

To do this, they needed to build a system that make a mechanical limb start moving in the right direction a split second ahead of time to make up for the delay. They implanted electrodes into the motor cortex of rhesus macaque monkeys who were trained to move a cursor on a screen by thinking about moving their own arms. The movements of the cursor approximated the movements of a robotic arm, with a delay built in that could be adjusted for the experiment.

Nicholas Hatsopoulos, PhD

Nicholas Hatsopoulos, chair of the Committee on Computational Neuroscience and senior author of the study, said that they found they could minimize the effects of delay in this system by factoring in two things: the cell activity corresponding to where the monkey is thinking about moving its arm next, and the recent movements of the arm itself.

“You might ask, ‘How can you possibly predict what he’s going to do?’” he said. “But the cell activity does tell you what the monkey is thinking about in the future.

In other words, the neural impulses indicate where it wants to move next, not where it is now. The real-world physics of moving a limb through space also help.

“From one moment to the next you can’t make completely unpredictable movements,” Hatsopoulos said. “Because of inertia you can’t go from here to here instantaneously, or you can’t reverse direction instantaneously. You’re moving a physical system, so we can account for that.”

By experimenting with various delay times, they found that they could project future movements up to 200 ms before performance started to tail off. The best results came when the amount of time projected into the future matched the delay.

Hatsopoulos said that while brain-machine interfaces are still in the research stage, they have the potential to improve prosthetic limbs and help patients with neurological conditions like ALS, which affect motor function. He and his team have already conducted a clinical trial with an ALS patient who was able to move a computer cursor via a brain-machine interface.

Barring further federal cuts to research funding, and with more support for projects like President Obama’s BRAIN Initiative to map neurological functions, within 5-10 years such systems could become less science fiction, and more of a reality.