Cluster of Excellence

Cognitive Interaction Technology

You are here

Communicating with the World through Brain Waves

06.12.2017

In May 2017, the research project “M³S – moderne Mensch-Maschine Schnittstelle” [Modern Human-Machine Interface] was launched. The project is being funded through 30 April 2020 by the European Regional Development Fund with a total of 1.5 million Euro. M³S is coordinated by Professor Dr. Ivan Volosyak, professor of biomedical sciences and engineering at the Rhine-Waal University of Applied Sciences. Volosyak’s team is working together with CITEC researchers Marc Hesse and Michael Adams, both from Professor Dr. Ulrich Rückert’s research group “Cognitronics and Sensor Systems.” The researchers are working on an interface between the brain and computer. In the future, the system is meant to be controllable directly via brain wave measurements. Following the kick-off event in Kleve, Germany, the first meeting was held in Bielefeld on 16 November.

“In the M3S project, we are developing an enhanced communications technology that helps people with physical disabilities to interact with the world around them,” says Marc Hesse, a member of the Cognitronics and Sensor Systems research group, which is led by Professor Dr. Ulrich Rückert. “For our research, we draw upon brain-machine interface technology. This technology measures brain activity using EEG (electroencephalography) and converts the data into commands in real time using a brain-machine interface.” In addition to the healthcare sector, the project partners from M3S also plan to use their system in the entertainment and gaming industries. One potential application is, for instance, a hands-free control for computer games. The system could also be used as a head-up display in the automotive industry, or in household activities.

Brain-machine interfaces (BMI) have the potential to take root in different industry branches as an assistive or even entertaining technology. Volosyak’s team wants to make the interface more robust and simplify the application. This will allow the researchers to reduce the training time needed to learn to use the interface. “We combine the brain-machine interface with eye control and a screen that has been specially conceptualized for the BMI-system,” says Hesse. “This will hopefully enable us to help us to bring the technology, which is still very young, closer to commercial viability in many different industrial sectors.” Ivan Volosyak’s team is specialized in brain-computer and brain-machine interfaces that make use of the normal activity in the human brain.

On conventional screens, several visual stimuli, such as different blinking boxes with constant frequencies, are displayed at the same time but separate from each other. By simply looking at a certain box, the user can select the desired command. When looking at something, brain waves modulate corresponding to the frequency being viewed. The accompanying EEG can classify a suitable command in real time.

CITEC researchers are working on resource-efficient processing of the signals, with the aim of reducing latency while at the same time ensuring the most precise calculations possible. Users are meant to be able to use the technology as easily as possible, which should lead to stronger acceptance of the system. In addition to this, Rückert’s team is working on an intuitive graphic user interface and on integrating the system into a Smart Home. The Cognitronics and Sensor Systems research group is taking the lead role on developing of the electronics for the display screen needed for the BMI system. The goal of the development is to create a controller for an LED Array that can include up to 200 different frequency areas. The user can select from among 200 different operation options, enabling them to intuitively operate the respective system. Users of conventional systems are quickly confused, their eyes get tired, or they get dizzy: with the M3S project, the researchers are working to exclude such negative side effects.

Further partners on this project include Polyoptics GmbH, which is developing the LED Array, and Mediablix-IIT GmbH. Mediablix-IIT GmbH is implementing the synchronized recording between stationary and mobile EEG and eye tracking measuring systems. Based on this, classification and evaluation procedures for the recorded data are supplied.

By the end of the project, the goal is to have a hybrid controller system that can be used with the latest wearable technologies, such as Google Glass, Microsoft HoloLens, or Oculus Rift. By combining this with other ready-for-market innovations that are already, like an eye control system and improved signal processing algorithms, the M³S project stands to solve the key remaining unresolved issues of BMI technology.