Brain-computer interfaces (BCIs) translate neural recordings into signals that may be used for communication and/or the control of neuroprosthetic devices. Research in this domain poses interesting challenges to machine learning, because data is typically scarce, noisy, and non-stationary [ ]. Furthermore, good decoding algorithms are contingent on domain knowledge that is not readily available and difficult to incorporate into traditional statistical methods. Accordingly, we employ machine-learning methods to study neural processes involved in BCI-control and use these insights to develop novel decoding algorithms and enhance experimental paradigms.

Machine learning algorithms for brain-state decoding

A crucial aspect of our work is the development of algorithms for real-time brain-state decoding. Building upon our experience in Bayesian inference, we have developed a graphical model decoding framework for ERP-based visual speller systems [ ]. This framework incorporates prior information on letter frequencies into the decoding process, thereby enhancing decoding performance. Furthermore, we were the first to successfully apply the framework of multi-task learning to the domain of BCIs [ ]. As the signal characteristics used by subjects to control a BCI share common aspects, the incorporation of data from previously recorded subjects substantially decreases calibration time and enhances overall decoding performance. Besides working on methods for real-time decoding, we also develop tools to investigate the neural basis of disorders of cognition. This is essential to understand how the diseased brain differs from the one of healthy subjects, which has implications for the design of BCI systems for patient populations. Building upon the framework of Causal Bayesian Networks, we were the first to provide a comprehensive set of causal interpretation rules for neuroimaging studies [ ]. Extending this line of work, we have developed a causal inference method that is able to detect causal relations between two brain processes, even in the presence of latent confounders [ ].

Brain-computer interfaces for communication

Building upon our machine-learning methods, we have investigated the neural basis of the ability to operate a BCI in healthy subjects and in patient populations. We could show that the configuration of large-scale cortical networks, as represented in high-frequency gamma-oscillations of the brain's electromagnetic field, influences a subject's ability to communicate with a BCI [ ]. This work has been awarded with the 2011 Annual Brain-Computer Interface Research Award. Building upon these insights, we have developed a novel class of BCIs for patients in late stages of amyotrophic lateral sclerosis [ ].

Brain-computer interfaces for rehabilitation

While BCIs were initially conceived as communication devices for the severely disabled, we have argued that they can also be used for stroke rehabilitation [ ]. By combining a BCI with a seven degrees-of-freedom robotic arm, that serves an exoskeleton, we could show a brain-controlled rehabilitation robot supports patient with chronic stroke in self-regulation of sensorimotor brain rhythms [ ]. The concept of brain-controlled rehabilitation robotics can be extended to systems that monitor patients' learning progress adapt the rehabilitation exercise in real-time [ ].

Objective. Brain–computer interface (BCI) systems are often based on motor- and/or sensory processes that are known to be impaired in late stages of amyotrophic lateral sclerosis (ALS). We propose a novel BCI designed for patients in late stages of ALS that only requires high-level cognitive processes to transmit information from the user to the BCI. Approach. We trained subjects via EEG-based neurofeedback to self-regulate the amplitude of gamma-oscillations in the superior parietal cortex (SPC). We argue that parietal gamma-oscillations are likely to be associated with high-level attentional processes, thereby providing a communication channel that does not rely on the integrity of sensory- and/or motor-pathways impaired in late stages of ALS. Main results. Healthy subjects quickly learned to self-regulate gamma-power in the SPC by alternating between states of focused attention and relaxed wakefulness, resulting in an average decoding accuracy of 70.2%. One locked-in ALS patient (ALS-FRS-R score of zero) achieved an average decoding accuracy significantly above chance-level though insufficient for communication (55.8%). Significance. Self-regulation of gamma-power in the SPC is a feasible paradigm for brain–computer interfacing and may be preserved in late stages of ALS. This provides a novel approach to testing whether completely locked-in ALS patients retain the capacity for goal-directed thinking.

2012

Subjects operating a brain-computer interface (BCI) based on sensorimotor rhythms exhibit large variations in performance over the course of an experimental session. Here, we show that high-frequency gamma-oscillations, originating in fronto-parietal networks, predict such variations on a trial-to-trial basis. We interpret this nding as empirical support for an in uence of attentional networks on BCI-performance via modulation of the sensorimotor rhythm.

Subjects operating a brain–computer interface (BCI) based on sensorimotor rhythms exhibit large variations in performance over the course of an experimental session. Here, we show that
high-frequency γ-oscillations, originating in fronto-parietal networks, predict such variations on a trial-to-trial basis. We interpret this finding as empirical support for an influence of attentional networks on BCI performance via modulation of the sensorimotor rhythm.

We report on the development and online testing of an electroencephalogram-based brain–computer interface (BCI) that aims to be usable by completely paralysed users—for whom visual or motor-system-based BCIs may not be suitable, and among whom reports of successful BCI use have so far been very rare. The current approach exploits covert shifts of attention to auditory stimuli in a dichotic-listening stimulus design. To compare the efficacy of event-related potentials (ERPs) and steady-state auditory evoked potentials (SSAEPs), the stimuli were designed such that they elicited both ERPs and SSAEPs simultaneously. Trial-by-trial feedback was provided online, based on subjects' modulation of N1 and P3 ERP components measured during single 5 s stimulation intervals. All 13 healthy subjects were able to use the BCI, with performance in a binary left/right choice task ranging from 75% to 96% correct across subjects (mean 85%). BCI classification was based on the contrast between stimuli in the attended stream and stimuli in the unattended stream, making use of every stimulus, rather than contrasting frequent standard and rare 'oddball' stimuli. SSAEPs were assessed offline: for all subjects, spectral components at the two exactly known modulation frequencies allowed discrimination of pre-stimulus from stimulus intervals, and of left-only stimuli from right-only stimuli when one side of the dichotic stimulus pair was muted. However, attention modulation of SSAEPs was not sufficient for single-trial BCI communication, even when the subject's attention was clearly focused well enough to allow classification of the same trials via ERPs. ERPs clearly provided a superior basis for BCI. The ERP results are a promising step towards the development of a simple-to-use, reliable yes/no communication system for users in the most severely paralysed states, as well as potential attention-monitoring and -training applications outside the context of assistive technology.

In recent work, we have provided evidence that fronto-parietal γ-oscillations of the electromagnetic field of the brain modulate the sensorimotor-rhythm. It is unclear, however, what impact this effect may have on explaining and addressing within-subject performance variations of brain-computer interfaces (BCIs). In this paper, we provide evidence that on a group-average classification accuracies in a two-class motor-imagery paradigm differ by up to 22.2% depending on the state of fronto-parietal γ-power. As such, this effect may have a large impact on the design of future BCI-systems. We further investigate whether adapting classification procedures to the current state of γ-power improves classification accuracy, and discuss other approaches to exploiting this effect.

In recent work, we have provided evidence that fronto-parietal γ-range oscillations are a cause of within-subject performance variations in brain-computer interfaces (BCIs) based on motor-imagery. Here, we explore the feasibility of using neurofeedback of fronto-parietal γ-power to induce a mental state that is beneficial for BCI-performance. We provide empirical evidence based on two healthy subjects that intentional attenuation of fronto-parietal γ-power results in an enhanced resting-state sensorimotor-rhythm (SMR). As a large resting-state amplitude of the SMR has been shown to correlate with good BCI-performance, our approach may provide a means to reduce performance variations in BCIs.

The combination of brain–computer interfaces (BCIs) with robot-assisted physical therapy constitutes a promising approach to neurorehabilitation of patients with severe hemiparetic syndromes caused by cerebrovascular brain damage (e.g. stroke) and other neurological conditions. In such a scenario, a key aspect is how to reestablish the disrupted sensorimotor feedback loop. However, to date it is an open question how artificially closing the sensorimotor feedback loop influences the decoding performance of a BCI. In this paper, we answer this issue by studying six healthy subjects and two stroke patients. We present empirical evidence that haptic feedback, provided by a seven degrees of freedom robotic arm, facilitates online decoding of arm movement intention. The results support the feasibility of future rehabilitative treatments based on the combination of robot-assisted physical therapy with BCIs.

A neurorehabilitation approach that combines robot-assisted active physical therapy and Brain-Computer Interfaces (BCIs) may provide an additional mileage with respect to traditional rehabilitation methods for patients with severe motor impairment due to cerebrovascular brain damage (e.g., stroke) and other neurological conditions. In this paper, we describe the design and modes of operation of a robot-based rehabilitation framework that enables artificial support of the sensorimotor feedback loop. The aim is to increase cortical plasticity by means of Hebbian-type learning rules. A BCI-based shared-control strategy is used to drive a Barret WAM 7-degree-of-freedom arm that guides a subject's arm. Experimental validation of our setup is carried out both with healthy subjects and stroke patients. We review the empirical results which we have obtained to date, and argue that they support the feasibility of future rehabilitative treatments employing this novel approach.

Robust dry EEG electrodes are arguably the key to making EEG Brain-Computer Interfaces (BCIs) a practical technology. Existing studies on dry EEG electrodes can be characterized by the recording method (stand-alone dry electrodes or simultaneous recording with wet electrodes), the dry electrode technology (e.g. active or passive), the paradigm used for testing (e.g. event-related potentials), and the measure of performance (e.g. comparing dry and wet electrode frequency spectra). In this study, an active-dry electrode prototype is tested, during a motor-imagery task, with EEG-BCI in mind. It is used simultaneously with wet electrodes and assessed using classification accuracy. Our results indicate that the two types of electrodes are comparable in their performance but there are improvements to be made, particularly in finding ways to reduce motion-related artifacts.

Motor-imagery-based brain-computer interfaces (BCIs) commonly use the common spatial pattern filter (CSP) as preprocessing step before feature extraction and classification. The CSP method is a supervised algorithm
and therefore needs subject-specific training data for calibration, which is very time consuming to collect. In order to reduce the amount of calibration data that is needed for a new subject, one can apply multitask (from now on called multisubject) machine learning techniques to the preprocessing phase. Here, the goal of multisubject learning is to learn a spatial filter for a new subject
based on its own data and that of other subjects. This paper outlines the details of the multitask CSP algorithm and shows results on two data sets. In certain subjects a clear improvement can be seen, especially when the number of training trials is relatively low.

International Journal of Bioelectromagnetism, 13(3):115-116, September 2011 (article)

Abstract

While research on brain-computer interfacing (BCI) has seen tremendous progress in recent years, performance still varies substantially between as well as within subjects, with roughly 10 - 20% of subjects being incapable of successfully operating a BCI system. In this short report, I argue that this variation in performance constitutes one of the major obstacles that impedes a successful commercialization of BCI systems. I review the current state of research on the neuro-physiological causes of performance variation in BCI, discuss recent progress and open problems, and delineate potential research programs for addressing this issue.

Analyzing neural signals and providing feedback in real-time is one of the core characteristics of a brain-computer interface (BCI). As this feature may be employed to induce neural plasticity, utilizing BCI-technology for therapeutic purposes is increasingly gaining popularity in the BCI-community. In this review, we discuss the state-of-the-art of research on this topic, address the principles of and challenges in inducing neural plasticity by means of a BCI, and delineate the problems of study design and outcome evaluation arising in this context. The review concludes with a list of open questions and recommendations for future research in this field.

Gamma oscillations of the electromagnetic field of the brain are known to be involved in a variety of cognitive processes, and are believed to be fundamental for information processing within the brain. While gamma oscillations have been shown to be correlated with brain rhythms at different frequencies, to date no empirical evidence has been presented that supports a causal influence of gamma oscillations on other brain rhythms. In this work, we study the relation of gamma oscillations and the sensorimotor rhythm (SMR) in healthy human subjects using electroencephalography. We first demonstrate that modulation of the SMR, induced by motor imagery of either the left or right hand, is positively correlated with the power of frontal and occipital gamma oscillations, and negatively correlated with the power of centro-parietal gamma oscillations. We then demonstrate that the most simple causal structure, capable of explaining the observed correlation of gamma oscillations and the SMR, entails a causal influence of gamma oscillations on the SMR. This finding supports the fundamental role attributed to gamma oscillations for information processing within the brain, and is of particular importance for brain–computer interfaces (BCIs). As modulation of the SMR is typically used in BCIs to infer a subject's intention, our findings entail that gamma oscillations have a causal influence on a subject's capability to utilize a BCI for means of communication.

This paper reviews several critical issues facing signal processing for brain–computer interfaces (BCIs) and suggests several recent approaches that should be further examined. The topics were selected based on discussions held during the 4th International BCI Meeting at a workshop organized to review and evaluate the current state of, and issues relevant to, feature extraction and translation of field potentials for BCIs. The topics presented in this paper include the relationship between electroencephalography and electrocorticography, novel features for performance prediction, time-embedded signal representations, phase information, signal non-stationarity, and unsupervised adaptation.

2010

Journal of NeuroEngineering and Rehabilitation, 7(34):1-4, July 2010 (article)

Abstract

Even though feedback is considered to play an important role in learning how to operate a brain-computer interface (BCI), to date no significant influence of feedback design on BCI-performance has been reported in literature. In this work, we adapt a standard motor-imagery BCI-paradigm to study how BCI-performance is affected by biasing the belief subjects have on their level of control over the BCI system. Our findings indicate that subjects already capable of operating a BCI are impeded by inaccurate feedback, while subjects normally performing on or close to chance level may actually benefit from an incorrect belief on their performance level. Our results imply that optimal feedback design in BCIs should take into account a subject&lsquo;s current skill level.

Brain-Computer Interfaces based on electrocorticography (ECoG) or electroencephalography (EEG), in combination with robot-assisted active physical therapy, may support traditional rehabilitation procedures for patients with
severe motor impairment due to cerebrovascular brain damage caused by stroke. In this short report, we briefly review the state-of-the art in this exciting new field,
give an overview of the work carried out at the Max Planck Institute for Biological Cybernetics and the University of T{\"u}bingen, and discuss challenges that need to be addressed in order to move from basic research to clinical studies.

Brain-Computer Interfaces (BCI) that rely upon epidural electrocorticographic signals may become a promising tool for neurorehabilitation of patients with severe hemiparatic syndromes due to cerebrovascular, traumatic or tumor-related brain damage. Here, we show in a patient-based feasibility study that online classification of arm movement intention is possible. The intention to move or to rest can be identified with high accuracy (~90 %), which is sufficient for BCI-guided neurorehabilitation. The observed spatial distribution of relevant features on the motor cortex indicates that cortical reorganization has been induced by the brain lesion. Low- and high-frequency components of the electrocorticographic power spectrum provide complementary information towards classification of arm movement intention.

Brain-computer interfaces (BCIs) are limited
in their applicability in everyday settings
by the current necessity to record subjectspecific
calibration data prior to actual use
of the BCI for communication. In this paper,
we utilize the framework of multitask
learning to construct a BCI that can be used
without any subject-specific calibration process.
We discuss how this out-of-the-box BCI
can be further improved in a computationally
efficient manner as subject-specific data
becomes available. The feasibility of the approach
is demonstrated on two sets of experimental
EEG data recorded during a standard
two-class motor imagery paradigm from
a total of 19 healthy subjects. Specifically,
we show that satisfactory classification results
can be achieved with zero training data,
and combining prior recordings with subjectspecific
calibration data substantially outperforms
using subject-specific data only. Our
results further show that transfer between
recordings under slightly different experimental
setups is feasible.

2009

In EMBC 2009, pages: 5304-5307, (Editors: Y Kim and B He and G Worrell and X Pan), IEEE Service Center, Piscataway, NJ, USA, 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, September 2009 (inproceedings)

Abstract

Implicit Wiener series are a powerful tool to build
Volterra representations of time series with any degree of nonlinearity.
A natural question is then whether higher order
representations yield more useful models. In this work we
shall study this question for ECoG data channel relationships
in epileptic seizure recordings, considering whether quadratic
representations yield more accurate classifiers than linear ones.
To do so we first show how to derive statistical information on
the Volterra coefficient distribution and how to construct seizure
classification patterns over that information. As our results
illustrate, a quadratic model seems to provide no advantages
over a linear one. Nevertheless, we shall also show that the
interpretability of the implicit Wiener series provides insights
into the inter-channel relationships of the recordings.

Spatial filtering (SF) constitutes an integral part of building EEG-based braincomputer interfaces (BCIs). Algorithms frequently used for SF, such as common spatial patterns (CSPs) and independent component analysis, require labeled training data for identifying filters that provide information on a subject‘s intention, which renders these algorithms susceptible to overfitting on artifactual EEG components. In this study, beamforming is employed to construct spatial filters that extract EEG sources originating within predefined regions of interest within the brain. In this way, neurophysiological knowledge on which brain regions are relevant for a certain experimental paradigm can be utilized to construct unsupervised spatial filters that are robust against artifactual EEG components. Beamforming is experimentally compared with CSP and Laplacian spatial filtering (LP) in a two-class motor-imagery paradigm. It is demonstrated that beamforming outperforms CSP and LP on noisy datasets, while CSP and beamforming perform almost equally well on datasets with few artifactual trials. It is concluded that beamforming constitutes an alternative method for SF that might be particularly useful for BCIs used in clinical settings, i.e., in an environment where artifact-free datasets are difficult to obtain.

EEG connectivity measures could provide a new type of feature space for inferring a subject&amp;amp;lsquo;s intention in Brain-Computer Interfaces (BCIs). However, very little is known on EEG connectivity patterns for BCIs. In this study, EEG connectivity during motor imagery (MI) of the left and right is investigated in a broad frequency range across the whole scalp by combining Beamforming with Transfer Entropy and taking into account possible volume conduction effects. Observed connectivity patterns indicate that modulation intentionally induced by MI is strongest in the gamma-band, i.e., above 35 Hz. Furthermore, modulation between MI and rest is found to be more pronounced than between MI of different hands. This is in contrast to results on MI obtained with bandpower features, and might provide an explanation for the so far only moderate success of connectivity features in BCIs. It is concluded that future studies on connectivity based BCIs should focus on high frequency bands and con
side
r ex
peri
mental paradigms that maximally vary cognitive demands between conditions.

2008

We address two shortcomings of the common spatial patterns (CSP) algorithm for spatial filtering in the context of brain--computer interfaces (BCIs) based on electroencephalography/magnetoencephalography (EEG/MEG): First, the question of optimality of CSP in terms of the minimal achievable classification error remains unsolved. Second, CSP has been initially proposed for two-class paradigms. Extensions to multiclass paradigms have been suggested, but are based on heuristics. We address these shortcomings in the framework of information theoretic feature extraction (ITFE). We show that for two-class paradigms, CSP maximizes an approximation of mutual information of extracted EEG/MEG components and class labels. This establishes a link between CSP and the minimal classification error. For multiclass paradigms, we point out that CSP by joint approximate diagonalization (JAD) is equivalent to independent component analysis (ICA), and provide a method to choose those independent components (ICs) that approximately
maximize mutual information of ICs and class labels. This eliminates the need for heuristics in multiclass CSP, and allows incorporating prior class probabilities. The proposed method is applied to the dataset IIIa of the third BCI competition, and is shown to increase the mean classification accuracy by 23.4% in comparison to multiclass CSP.

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems