Login using

You can login by using one of your existing accounts.

We will be provided with an authorization token (please note: passwords are not shared with us) and will sync your accounts for you. This means that you will not need to remember your user name and password in the future and you will be able to login with the account you choose to sync, with the click of a button.

Large efforts are currently being made to develop and improve online analysis of brain activity which can be used, e.g., for brain–computer interfacing (BCI). A BCI allows a subject to control a device by willfully changing his/her own brain activity. BCI therefore holds the promise as a tool for aiding the disabled and for augmenting human performance. While technical developments obviously are important, we will here argue that new insight gained from cognitive neuroscience can be used to identify signatures of neural activation which reliably can be modulated by the subject at will. This review will focus mainly on oscillatory activity in the alpha band which is strongly modulated by changes in covert attention. Besides developing BCIs for their traditional purpose, they might also be used as a research tool for cognitive neuroscience. There is currently a strong interest in how brain-state fluctuations impact cognition. These state fluctuations are partly reflected by ongoing oscillatory activity. The functional role of the brain state can be investigated by introducing stimuli in real-time to subjects depending on the actual state of the brain. This principle of brain-state dependent stimulation may also be used as a practical tool for augmenting human behavior. In conclusion, new approaches based on online analysis of ongoing brain activity are currently in rapid development. These approaches are amongst others informed by new insight gained from electroencephalography/magnetoencephalography studies in cognitive neuroscience and hold the promise of providing new ways for investigating the brain at work.

Introduction

In recent years there has been a strong increase in the interest in characterizing brain activity online, for instance in the context of brain–computer interfaces (BCIs). A typical BCI setup allows for the online characterization of ongoing brain activity recorded by electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), or intracranial recordings. The ongoing brain activity is used for controlling various devices through computer interfaces. Examples of applications are computer gaming, communication devices for highly impaired patients, rehabilitation, control of artificial limbs, and neuro-feedback (Lebedev and Nicolelis, 2006; Pfurtscheller and Neuper, 2006; Allison et al., 2007; Daly and Wolpaw, 2008; Van Gerven et al., 2009; Zander and Kothe, 2011).

To construct a robust BCI it is essential to make use of signatures of brain activity that can be modulated at will and quantified using short segments of data. In recent years there has been a rapid development in the application of brain imaging techniques to address cognitive questions. From these applications we have gained insight into which signal modulations are robust given a certain task. In particular it is becoming clear that oscillatory brain activity in the lower frequency range (<30 Hz) is reliably modulated in various cognitive tasks and can be detected with a reasonable signal-to-noise ratio using EEG and MEG (Hari and Salmelin, 1997; Klimesch et al., 2006; Jensen and Mazaheri, 2010).

While cognitive neuroscience can inform the development of BCI, the reverse might also hold (Blankertz et al., 2010). First of all, BCI forces researchers to focus on the strongest and most robust task-dependent modulations of brain signals. This could serve to ensure that an empirical investigation does not “get stuck” on a working hypothesis pertaining to aspects of the data being relatively weak. Also when developing a BCI it is essential to control for various confounds such as task difficulty in order to get reliable signals. During this process one often stumbles on new experimental questions pertaining to fundamental aspects that might not have been addressed before. See for instance (Bahramisharif et al., 2011) in which modulations of brain activity by covert attention prompted a question on how these changes were modulated by eccentricity. Second, online detection of brain activity can be used to train subjects’ ability to modulate their own ongoing brain activity and investigate the consequences for cognitive processing. Third, online detection of brain activity allows for presenting stimuli to the subject timed by the online measured brain activity. This is here referred to as brain-state dependent stimulation (BSDS) and provides a new avenue for investigating the neuronal substrate of cognition. In this paper we will give some examples in which this approach has been applied to gain new insight into human cognition. Furthermore we will propose ideas on how BSDS can be used in future settings to provide new neuroscience knowledge and augment human behavior.

The Functional Role of Oscillatory Brain Activity in the Alpha Band

The human brain produces oscillatory activity in various frequency bands both during rest and while performing cognitive tasks. What is the functional role of oscillatory brain activity? We will first focus on activity in the alpha band (8–13 Hz) since it has a strong signal-to-noise ratio and is often used as a control signal in BCI setups (Pfurtscheller and Neuper, 2006; Lou et al., 2008; Van Gerven et al., 2009; Boord et al., 2010). The alpha rhythm was first reported by Hans Berger in the late 1920s (Berger, 1929). When subjects are resting, it is by far the strongest electrophysiological signal that can be recorded from the human scalp. MEG research using source modeling has localized the sources of the alpha activity to parieto-occipital regions and primary sensorimotor regions around the central sulcus (Hari and Salmelin, 1997; Jensen and Vanni, 2002). Given that alpha activity is strongest when people rest and close their eyes, it has long been thought that it is related to brain states where few mental operations are occurring and thus labeled the “idling rhythm” (Pfurtscheller et al., 1996). Over the last decade, the idling hypothesis has been challenged (Klimesch et al., 2006; Palva and Palva, 2007; Thut and Miniussi, 2009; Jensen and Mazaheri, 2010). In particular it has been demonstrated that alpha activity actually can increase with cognitive load (Jensen et al., 2002; Scheeringa et al., 2009; Haegens et al., 2010; Khader et al., 2010; Meeuwissen et al., 2010). For instance, during working memory retention when no visual input are introduced, the alpha activity increases with the number of items that the subject has to retain (Jensen et al., 2002; Jensen, 2006; Tuladhar et al., 2007; Scheeringa et al., 2009). Other studies have used cognitive tasks which directly manipulate the engagement and disengagement of certain regions (Jokisch and Jensen, 2007; Rihs et al., 2007; Van Der Werf et al., 2008; Sauseng et al., 2009; Haegens et al., 2010; Romei et al., 2010; Freunberger et al., 2011). For instance, it is well established that alpha activity is depressed when covert attention is directed toward visual stimuli. However, when attention is directed toward auditory stimuli, the alpha activity over occipital areas increases (Fu et al., 2001). Other studies in which attention was directed toward the left or right visual hemifield have established that alpha activity is depressed in the hemisphere contralateral to the attended hemifield, while it often increases ipsilaterally (Worden et al., 2000; Thut et al., 2006; Rihs et al., 2007; Van Gerven et al., 2009; Cosmelli et al., 2011; Gould et al., 2011). These findings support the notion that the alpha activity reflects inhibition of regions not involved in a given task (Klimesch et al., 2006). This inhibition will serve to allocate resources to areas involved in the actual processing. Indeed it has been suggested that oscillatory activity subserves a gating function (Lopes Da Silva, 1991). In the light of recent empirical findings, this view has been pushed further in the “gating by inhibition hypothesis” (Jensen and Mazaheri, 2010). According to this hypothesis, task-irrelevant areas are inhibited by the alpha activity in order to actively gate information to the task-relevant areas. By this principle the alpha activity can shape the functional architecture of the working brain. This idea is supported by several recent findings which have demonstrated that if the alpha activity is not strong enough in the task-irrelevant areas, then performance is suboptimal. For instance, in a recent MEG study, oscillatory brain activity was investigated in a somatosensory working memory task (Haegens et al., 2010). Stimuli were delivered to the right hand and thus engaged the left hemisphere. Interestingly, behavioral performance correlated with the magnitude of widespread right hemispheric alpha activity from the retention interval (between sample and probe) including right somatosensory areas (Figure 1). Support for the notion that the alpha activity serves to causally inhibit was found using paradigms engaging respectively the left or the right hemisphere together with TMS and EEG combined. Entraining the task-irrelevant hemisphere by TMS pulses at alpha frequency serves to increase performance (Sauseng et al., 2009; Romei et al., 2010).

FIGURE 1

Figure 1. Alpha activity in right sensorimotor areas predicts performance in a somatosensory working memory task engaging left hemisphere areas. This implies that functional inhibition of the right somatosensory cortex by alpha activity is predictive of successful performance. (A) Patterns of electrical stimuli were presented to the right hand and probed 2 s later. (B) When comparing the 8- to 13-Hz alpha activity in the retention interval between sample and probe (correct–incorrect), the right hemisphere alpha activity was stronger for successful performance. (C) Time–frequency representations of power showed that the task-dependent effect was constrained to the alpha band in the retention interval. (D) The alpha sources predicting successful performance were localized in areas including right sensorimotor and parieto-occipital regions. Reproduced with permission from Haegens et al. (2010).

A recent long-term memory study investigated the encoding of word sequences (Figure 2). After the words were presented, subjects had to rehearse them for a brief period of 3.4 s. Later, the memory for the word sequences was tested. This allowed for investigating the rehearsal activity associated with successful memory encoding. The key finding was that the alpha activity must be strong in occipital areas in order to support memory encoding (Meeuwissen et al., 2010). This alpha activity is likely to suppress incoming visual information interfering with the memory task. In short, it is now clear that alpha activity plays an important role in neuronal processing. Optimal processing requires alpha activity in task-irrelevant areas. These studies have helped to gain considerable insight into which tasks are effective for modulating the alpha activity. Paradigms in which covert attention is directed to different modalities or spatial locations, seem to strongly modulate the alpha activity. This insight is now being used in the development of novel BCI (Kelly et al., 2005; Van Gerven et al., 2009; Van Gerven and Jensen, 2009; Bahramisharif et al., 2010a, 2011).

FIGURE 2

Figure 2. An increase in posterior alpha activity predicts successful long-term memory encoding. This suggests that functional inhibition of occipital cortex is a requirement for successful memory formation. (A) Subjects were instructed to rehearse visually presented word triplets during a 3.4-s period. Later memory for the triplets was tested. This allowed for characterizing the brain activity during the rehearsal interval predicting successful memory formation. (B) The alpha activity was stronger for Later Remembered compared to Later Forgotten triplets. (C) The sources of the alpha activity predicting memory encoding were localized in the occipital cortex. Reproduced with permission from Meeuwissen et al. (2010).

In conclusion, different cognitive tasks produce robust modulations in oscillatory activity in various frequency bands and regions. For EEG and MEG data it is particularly the activity in the lower frequency bands (<30 Hz) that is under voluntary control and has a strong signal-to-noise ratio and is thus most suited for BCI applications. Higher frequency ranges might be promising as well but this remains to be further explored.

Alpha Modulated by Attention as a Control Signal for BCI – Toward Two-Dimensional Control

Which tasks might be optimal for modulating the alpha activity in a BCI setting? A now classic EEG study demonstrated that posterior alpha is modulated by covert spatial attention as seen in Figure 3 (Worden et al., 2000). Subjects were required to fixate at the center of a screen while covertly attending to a symbol in either the lower left or right hemifield. This manipulation produces a strong depression in the alpha activity in the hemisphere contralateral the attended hemifield compared to the alpha activity in the ipsilateral hemisphere. Can this hemispheric lateralization in alpha activity serve as a control signal for BCI? This question has been investigated in an MEG experiment in which data were subjected to single trial classification (Van Gerven et al., 2009). In several subjects it was indeed possible to reliably decode the direction of attention in individual trials of about 1 s length. The principle has also been applied in an online BCI setup (Bahramisharif et al., 2010a). The aim of the subjects was to move the “background” of a scene to the left or the right while fixating at the center of the screen. By covertly attending to the left or the right, a few subjects succeeded in moving the background in the desired direction. Previous work has established that BCI gives the feeling of a control if performance is above 70% (Kubler et al., 2004). The performance of classifying the two directions of covert attention is reported to be up to 86% for the offline experiment (Van Gerven and Jensen, 2009), and 83% for the online case (Bahramisharif et al., 2010a). This provides a proof-of-principle that modulation of hemispheric alpha lateralization by covert attention can be used as a control signal for BCI.

FIGURE 3

Figure 3. Hemispheric alpha lateralization in response to left or right covert attention. This effect has later been tested as a control signal for BCI. When subjects were asked to attend to the right, left hemisphere alpha was depressed compared to when attention was directed to the left. The reverse effect was found over the right hemisphere. Reproduced with permission from Worden et al. (2000).

Not only do changes in covert attention in the horizontal plane modulate posterior alpha activity, so do vertical changes. In an EEG study subjects were asked to covertly direct their attention to eight different directions. When inspecting the topographical maps of the alpha activity, there were strong modulations according to the direction of attention (Rihs et al., 2007). When subjects were attending down, the posterior alpha activity was suppressed more so than when attention was directed up. This implies that the vertical direction of covert attention might be used for BCI control as well. In an MEG study in which subject were asked to covertly attend to a symbol in one of four directions (up, down, left, right) it was indeed possible to decode the attended direction in single trials (Van Gerven et al., 2009). This raises the possibility that an arbitrary direction of covert attention can be decoded by taking the topographic distribution of covert attention into account. This was tested in a setup in which subjects were asked to fixate centrally but covertly track the direction of a moving target. Using a sliding time window of 1700 ms it was possible to decode the direction of attention well above chance with a mean absolute deviation between the actual and predicted directions of about 60 degrees for the best case (Bahramisharif et al., 2010b). This error can be considerably reduced if smooth target movement over time is used as an additional constraint. This constraint can be mathematically expressed using a linear dynamical system where the predicted direction of attention not only depends on the alpha activity but also on the previous predicted direction. By virtue of this approach the mean absolute deviation decreased from 60 to 48 degrees for the best case. See Figure 4 for a visualization of these results.

FIGURE 4

Figure 4. The topography of the alpha activity is modulated by covert attention in different directions. This opens the possibility of making a BCI system which tracks attention in two dimensions. (A) Subjects were asked to covertly attend to one of eight directions for a 1300-ms period while fixating centrally. During this interval the alpha power was estimated using data from a 128 channel EEG system. The resulting topographies for the eight direction are shown. Both vertical and horizontal directions of covert attention clearly modulated the distribution in the alpha band. Reproduced with permission from Rihs et al. (2007). (B) In an MEG experiment subjects were asked to continuously track the direction of a target moving on a circle while fixating at the central cross. (C) The directions of attention were arbitrarily divided into 16 subparts and the respective alpha power was estimated using at 275 sensor MEG system. The topography of the alpha band activity is clearly modulated by the direction of attention. (D) The direction of covert attention was estimated from 1700 ms data segments. The true angle of the target could be predicted from the alpha activity alone with some error (top panel). After a smoothness constraint was applied (lower panel), the prediction error was much reduced. [(B,C,D), top panel] Reproduced with permission from Bahramisharif et al. (2010b).

This collection of studies demonstrates that modulations in alpha activity with covert attention can be used as a control signal for BCI. How robustly it works is under investigation; nevertheless, the initial findings are very promising. Using visual covert attention for BCI might have several advantages compared to other paradigms used such as motor imagery. First, concerns about EMG artifacts driving the BCI are less. Nevertheless, care has to be taken that eye-movements and blinks do not confound the signals of interest. Second, when constructing a BCI controlled cursor (“brain mouse”) it is quite a natural setup to have the cursor (or the background of a scene) move in the attended direction. Furthermore, the modulation of alpha activity by directing attention vertically allows for constructing a two-dimensional cursor control, something that only has been achieved in a few non-invasive BCI studies (Wolpaw and Mcfarland, 2004; Li et al., 2010). Future research will reveal how robust covert spatial attention is as a control signal for BCI.

It might as well be advantageous to use attention modulations in other modalities as a control signal for BCI. Shifting attention from the visual to the auditory modality is known to produce robust modulations of posterior alpha activity (Fu et al., 2001). Surprisingly, hemispheric alpha lateralization can also be modulated by directing the auditory attention to sounds presented to the left or the right (Kerlin et al., 2010). Finally, the alpha (and beta) activity produced in sensorimotor cortices is strongly modulated by attention. Shifting attention from the left to the right hand provides a very reliable modulation in hemispheric lateralization of the sensorimotor alpha activity (Jones et al., 2010; Van Ede et al., 2010, 2011; Haegens et al., 2011). In sum, various attention manipulations resulting in the modulation of oscillatory activity observed in cognitive studies hold the promise of providing robust control signals for BCI. In particular the shift in attention within and between sensory modalities could be used to increase the dimensionality of BCI control.

Electrophysiological Signals Reflecting the State of the Brain

So far we have addressed how cognitive neuroscience can inform BCI, but what might BCI teach us about the physiological underpinnings of human cognition? Recently there has been a growing interest in understanding the resting state of the brain and how it might impact neuronal processing (Silvanto et al., 2008). This is amongst others motivated by the observation that spontaneous fluctuations in brain activity often are much stronger in magnitude compared to stimulus induced activation. Based on this finding, the fMRI community has been investing the so-called resting state networks (Raichle and Snyder, 2007). The goal has been to uncover regions that are functionally coupled in terms of correlated BOLD fluctuations. Pre-stimulus fluctuations in the BOLD signals are not only informative for understanding functional coupling, they have also been shown to correlate with cognitive processing (Otten et al., 2002; Hesselmann et al., 2008,b; Sadaghiani et al., 2009; Park and Rugg, 2010). The role of spontaneous brain activity in cognitive processing has also been investigated using EEG and MEG recordings in humans. This is mainly done by quantifying how pre-stimulus activity correlates with subsequent perception (Ergenoglu et al., 2004; Linkenkaer-Hansen et al., 2004; Hanslmayr et al., 2005; Van Dijk et al., 2008; Guderian et al., 2009; Musso et al., 2010; Zhang and Ding, 2010). In line with the functional inhibition hypothesis of alpha, it has been demonstrated that strong pre-stimulus alpha activity has a negative impact on perception (Ergenoglu et al., 2004; Van Dijk et al., 2008; Mathewson et al., 2009). Furthermore, somatosensory stimulus detection is modulated by alpha activity produced around the central sulcus (Linkenkaer-Hansen et al., 2004; Zhang and Ding, 2010). These findings do not only hold for hard-to-detect stimuli. Pre-stimulus oscillatory activity was recently shown to affect the performance in Go–noGo task (Mazaheri et al., 2009). Subjects were presented with digits (“1” to “9”). They had to press a button for each digit, but withhold the response when digit “5” was presented. Erroneous button presses were preceded by higher pre-stimulus alpha activity. The higher alpha activity might slow down the processing of the visual stimuli, thus preventing that a stop-signal for the motor system is generated in time.

These studies show that the state of the brain has a strong impact on cognitive processing. Given that the brain-state modulates perception, it implies that how we perceive and remember the world around us is dependent on fluctuations in ongoing brain activity. Thus by presenting stimuli depending on the brain state it should be possible to manipulate perception and memory.

Brain-State Dependent Stimulation

The neuronal substrate of the spontaneous fluctuations of the brain state is reflected in electrophysiological activity that can be measured with EEG, MEG, or intracranial recordings. This raises the possibility of manipulating cognitive processing by taking the online brain activity into account. One way of doing this is by changing stimuli presented to the subject depending on an online characterization of the brain activity (Hartmann et al., 2011). This was done in a recent study in which single neuron firing was recorded from human patients with electrodes implanted in medial temporal lobe regions (Cerf et al., 2010), as shown in Figure 5. The electrodes were implanted in order to identify epileptic foci prior to surgery; however, the subjects consented to participate in a cognitive study as well. What Cerf and colleagues first did was to identify neurons that fired in response to a particular picture of a famous person (e.g., Marilyn Monroe or Josh Brolin). These pictures where then superimposed but a subject was for instance asked to attend to Monroe. If the Monroe cell increased its firing (and the Brolin cell decreased), the contrast of the Monroe picture was increased by the online setup at the expense of the Brolin picture. The key finding was, that by using this manipulation, subjects were able to intentionally manipulate the visual stimuli and thus their own percept. The findings suggest that stimulus specific neurons in the human MTL are under top-down control and that their firing can be modulated by attention.

FIGURE 5

Figure 5. Online control of the firing of temporal lobe neurons. (A) Electrodes allowing for detecting single unit firing were implanted in the temporal lobe of patients prior to surgery. The firing of multiple neurons was analyzed online. The output of the analysis controlled a visual display. (B) In the visual display photos of famous persons (e.g., Marilyn Monroe and Josh Brolin) were superimposed. Subjects were asked to attend to one of the persons (e.g., Monroe). If the neurons coding for Monroe (C) fired stronger, the Monroe face was made more visible by the control loop. The example in (B) shows eight trials in which the subject was able to make the Monroe picture visible by voluntarily controlling the firing of the temporal lobe neurons. (D) The visibility of the target photo averaged over several trials. Reproduced with permission from Cerf et al. (2010).

Brain-state dependent stimulation has been used to investigate the function of the phase of ongoing oscillations. Gho and Varela (1988) studied the role of alpha phase in the context of the “double flash” illusion using an EEG setup. When two visual stimuli are flashed with a slight delay, they can either be perceived as two flashes or as a moving light. In the EEG study the stimuli were delivered depending on the phase of the ongoing alpha activity. The findings demonstrated that the perception (two flashes versus a single moving light) could be somewhat manipulated depending on the alpha phase. In a more recent study the relationship between alpha phase and evoked responses was addressed (Kruglikov and Schiff, 2003). The phase of the ongoing alpha activity was characterized online and the auditory stimuli were then presented as a function of the phase. This allowed the authors to demonstrate that the magnitude of the P50 was influenced by alpha phase. These two studies suggest that human perception is not continuous, but rather discrete and clocked by ongoing oscillations (Vanrullen and Koch, 2003).

Spontaneous oscillations are not only observed at rest but also during sleep. Oscillations during sleep have been hypothesized to be involved in offline processing of information acquired during the day (Diekelmann and Born, 2010). For instance it has been proposed that slow wave oscillations reflect the reactivation of recently acquired information. This reactivation might support memory consolidation by recoding the neuronal representations in neocortical areas (Rasch and Born, 2007). To test this notion, a recent study investigated the consequences of perturbing slow wave activity. During sleep, EEG was used to record the ongoing brain activity. When slow wave activity increased above a certain threshold, a device started to play sounds. These sounds served to reduce the slow wave activity. After sleep the brain activation following the recall of memories recently acquired was tested. The study demonstrated that MTL activation probed by the memory recall was reduced following the slow wave intervention (Van Der Werf et al., 2009). Using this type of BSDS, it was concluded that slow wave sleep is causally related to memory formation.

This set of studies illustrates a few examples in which BSDS has been applied in order to gain new insight into the functional role of ongoing brain activity. Given the growing interest in the role of brain states for cognition, BSDS is likely to become a more frequently used tool in cognitive neuroscience.

Augmenting Human Behavior Using BCI

There are several approaches for using BCI-type setups to augment human performance (Parra et al., 2008). For instance, it is possible to online assess mental workload as reflected by neurophysiological activity using EEG. Using this measure of workload, the task demands can be adjusted in order to improve performance. This possibility has been investigated in the context of a virtual reality driving environment (Kohlmorgen et al., 2007). Another possibility is to detect errors committed in demanding response tasks using event-related negativity (ERN). The ERN is a frontal negative event-related potential following errors after button presses. In a visual task subjects were asked to discriminate between two visual stimuli by pressing one of two buttons. Since mistakes were followed by an increased ERN, this allowed for correcting erroneous responses and thus increasing performance by more than 20% (Parra et al., 2003).

Posterior alpha activity might also be used in designs to augment human behavior. The finding that visual perception is reduced with strong posterior alpha activity can be used in an online setup where difficult-to-detect stimuli are presented only when the alpha activity is low. This should increase the visual detection rate as compared to when the stimuli are presented without taking the brain state into account. Another possibility is to make use of the finding that memory rehearsal improves encoding when the posterior alpha activity is strong (Khader et al., 2010; Meeuwissen et al., 2010). This allows for predicting to what extent a given memory later can be remembered: If the online alpha activity suggests that an item might be forgotten, it should be presented again. In summary there are several ideas for how BSDS can help augment human behavior; however, many of these ideas need to be tested. The possibility of augmenting memory encoding could have many practical applications. One example would be computer-assisted learning of foreign vocabulary. If the online system could predict how well new word pairs are encoded, the system could be used to optimize learning. When developing these applications a valuable side product might be new insight gained in cognitive neuroscience. For instance, when developing online systems for augmenting memory encoding, we are bound to gain new insight into the brain activity necessary for successful memory formation.

Technical Aspects: Using Multisensor MEG and EEG Systems for BCI

How many sensors are required in a BCI setup and when is it sensible to use MEG rather than EEG? The answers to these questions are of course dependent on the particular application. For portable BCIs to be used in practical settings it is important to have relatively few sensors and a lightweight portable system. However, when BCI is applied as a tool for cognitive neuroscience, it is important to have as much information as possible available. For such applications, a multisensor system will allow for better separation of the various sources being active in different regions. While it has long been appreciated that EEG and MEG signals are a direct consequence of neuronal activity they still come with significant observability constraints (Lopes Da Silva, 1991). EEG/MEG recorded at the scalp will be dominated by a number of components that may either be artifactual or reflect synchronized processing in the underlying neural substrate. This processing may or may not be related to the task at hand. For instance, in the case of covert attention paradigms it is possible that two components are active at the same time: first, a more global alpha component, which is strongly modulated by the vigilance state (akin to the classical “idling” rhythm) and second, a more focal/discrete alpha component, which is modulated according to changes in covert attention. At the scalp, these two components will be mixed together. Hence the main objective when dealing with such macroscopic signals is to first separate the components and then examine their characteristics. MEG will be the most optimal tool given that the spatial smearing at the sensor level is much less compared to EEG (Hamalainen et al., 1993). Due to its biophysical properties, chiefly the fact that magnetic fields do not get distorted by inhomogeneous electrical properties of the volume conductor, MEG is better suited to separate and localize the different components. The isolation of relevant components increases specificity and enhances their signal-to-noise ratio whereas their localization is crucial in creating a link to invasive data and hence to the underlying physiology.

Performing BCI experiments with multisensor EEG and MEG systems is highly demanding given that large amounts of data need to be streamed and processed online. Additionally, one needs to be able to suppress environmental noise in real-time and have access to all the information required to perform online source reconstruction and artifact removal. There are now several initiatives working toward a standardized approached for data streaming and analysis including BCI2000, MatRiver, OpenVIBE (Schalk et al., 2004; Delorme et al., 2010; Renard et al., 2010; Hartmann et al., 2011). As an example for how such a system can work we will here describe an implementation developed using the FieldTrip toolbox (Oostenveld et al., 2011).

The foremost challenge for an online system is to be able to achieve fast streaming of large amounts of data (up to ∼300 sensors at 1 kHz sampling rates). Here we note that not only do MEG and EEG data channels need to be recorded but also EMG and EOG channels. In the case of MEG, reference channels allowing for real-time cancelation of environmental noise are often also collected. Secondly, one needs to access the head position with respect to the MEG sensors. This is necessary for source reconstruction and/or other data projections yielding higher signal-to-noise ratios and specificity. Online monitoring of the participant’s head position is highly desirable in order to check that the decoding in a BCI setup is not influenced by head movement. The process of localizing head positions is dependent on dipole fitting algorithms operating on the fields generated by coils attached to the subject’s head.

The details pertaining to the data writing process and the memory allocated to the online interface are specific to each system and different for systems of different manufacturers. In this light, the best option appears to be to devise a system which does not depend on the specific details of the interface provided by the manufacturers. This is precisely what has been developed at the in the FieldTrip toolbox, namely an access scheme to real-time data, which is platform and hardware independent. This access scheme is part of the open source FieldTrip software package and can be used for real-time streaming of MEG, EEG, and fMRI signals (http://fieldtrip.fcdonders.nl/development/realtime). The implementation of this real-time access scheme is illustrated schematically in Figure 6.

FIGURE 6

Figure 6. A schematic illustration of how real-time data access can be implemented. This example depicts the approached developed within the FieldTrip package working in conjunction with an MEG system. The MEG system acquisition software (Acq) writes data to a limited capacity memory segment set up on the acquisition computer (shared memory segment), which is the manufacturer-specific online interface. An external, manufacturer-independent buffer (FieldTrip buffer) is set up on the same or a different computer. An agent (acq2ft) continually monitors the shared memory segment for new data and copies them to the external buffer as soon as these become available. Once the data is in the external buffer they can be accessed and analyzed in Matlab for the purposes of the BCI. Crucially, the rapid copying of new data as soon as they become available to an external buffer deems the reverse-engineering of the MEG system data writing process superfluous.

The key motivation behind the real-time access scheme is to make the analysis software as simple and generic as possible, while at the same providing access to crucial information such as sensor locations and other metadata. This is achieved by involving an intermediate in-memory buffer (the “FieldTrip buffer”) to which data is streamed by manufacturer-specific software that runs on the acquisition machine. The buffer data can concurrently be read from by one or multiple analysis agents (e.g., Matlab scripts) on the same or a different computer. This is implemented by a software agent (acq2ft) which constantly monitors the shared memory segment to which the acquisition software (Acq) writes small blocks of data. The very first of these blocks contains a link to the setup file corresponding to the current recording, which is read and decoded by acq2ft and written to the header part of the buffer. Further blocks arrive in rapid succession and wrap around after some time. Since acq2ft copies the data to the buffer as soon as they arrive, the analysis agents do not have to worry about peculiarities of the timing: They can always access the metadata as well as a rather large window of continuously sampled data up to the latest block of samples. On top of that, acq2ft also decodes trigger channels and provides these to the analysis agents in the form of discrete markers. This facilitates the development of synchronous BCIs that are locked to particular trigger events.

In short, there are now useful developments simplifying the design and implementation of the analysis software for BCI and BSDS. These implementations help to abstract from the peculiarities of the hardware and will facilitate the development of standardized platforms.

Conclusion

There are several new exciting developments of BCIs and BSDS. Some of these developments are inspired by new insights from cognitive neuroscience. In particular it is becoming clear that oscillatory activity in the alpha band plays an important role in the allocation of neurocomputational resources. This is for instance evident in tasks where covert attention is modulated within or between modalities. This insight has opened up several new avenues for control signals for BCI, many of which are now actively being explored. BCI also allows for providing new insight into cognitive neuroscience. For instance BCI can be used as a tool to study the functional role of ongoing brain activity which reflects the state of the brain. This can be done using BSDS in which stimuli are introduced as a function of the measured oscillatory brain activity. Beyond providing new insight into the role of the oscillatory brain activity the approach also holds the promise of developing online systems for augmenting human performance.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs and the Netherlands Ministry of Education, Culture and Science the Netherlands Initiative Brain and Cognition, a part of the Organization for Scientific Research (NWO) under grant number 056-14-011, and The Netherlands Organization for Scientific Research (NWO): Innovational Research Incentive Schemes, VICI grant number: 453-09-002.