Alumni & Friends

Associate Professor Researching Ways to Control Devices with the Brain

It’s a cold, frosty morning, and your thoughts turn to snuggling under the covers with a cup of coffee while the house warms up.

Suddenly, your coffee machine kicks into gear just as the heat begins to rise in your home. No, it’s not your imagination. By merely thinking about it, your home is now cozier, your hot cup of java awaits in the kitchen, and you didn’t have to lift a finger.

The reality of controlling devices using only your brain is closer than you think, thanks to the research of Associate Professor Xiaopeng Zhao.

A drone is the first device Zhao and his students (graduate students Reza Abiri and Soheil Borhani and undergraduate Justin Kilmarx) have demonstrated the ability to pilot using brainwaves, and they are amazing spectators who have the opportunity to see them make the drone fly without a normal hand-held controller.

According to Zhao, the mind-controlled drone is an example of the amplification of brain computer interface (BCI), or using the brain to control computers and machines.

“We can monitor the brainwaves of a human subject using electroencephalography (EEG) sensors,” said Zhao. “We use the mind-controlled drone as a testbed to develop convenient, efficient, and accurate BCI paradigms. It is a great challenge to decode a human’s thoughts from EEG since the signals are very complex and subject to a huge amount of noises and artifacts.”

“Traditional algorithms based on sensorimotor rhythms require long training times are not feasible for all subjects. We aim to develop new algorithms to improve the usability of BCI for a broader public audience.”

To control a drone using the mind, an individual must first undergo a 10-minute training phase. While wearing an EEG headpiece, which places electrodes along the scalp, the individual imagines moving his or her hand to follow the trajectory of a cursor on a computer screen for a few minutes.

A machine learning model is developed using the training phase data to decode the imagined body kinematics from the individual’s EEG signals. After this, the subject is ready to control the drone.

When the individual imagines making the drone move forward or rotate, the brainwave signals are fed into the machine learning model. Once the individual’s intentions are decoded, the signals are sent to the drone through Wi-Fi in real time, and the drone will start flying.

The ability to control the drone is not an easy task and takes a lot of training and practice to master.

“The young minds seem to work better than the older ones,” said Zhao. “In my lab, the students control the drone much better than I do.”

Originally, research involving mind-controlled devices focused on medical applications for patients with motor disabilities or neurological disorders, but has evolved beyond healthcare needs.

“The exact technique used for the mind-controlled drone can be used to operate any other machine or device,” said Zhao. “Mind-controlled devices may change the way we live. For example, you might use your mind to adjust the temperature in a room or to make a cup of coffee.”

The drone is just one of the devices Zhao and his students can control with their mind, with robotic arms, cars, social robots, and computer games all on the slate.

Zhao is teaming up with other researchers to find beneficial ways to use the mind-controlled devices, including working with MABE Assistant Professor Subhadeep Chakraborty to develop and improve applications, partnering with UT Assistant Professor of Psychology Aaron Buss to improve performance through multimodal techniques, and even collaborating with researchers at the University of Kentucky to develop BCI tools for diagnosis of patients with cognition and attention deficits and train these patients for rehabilitation.

In the near future, as BCI research evolves and techniques improve, mind-controlled devices will be a normal part of life.

Improvements in healthcare, education, entertainment, communication, and yes, even for making the early morning routine run a little smoother, are just a few of the things that could be impacted.