Jun 21, 2016

Human-computer confluence refers to an invisible, implicit, embodied or even implanted interaction between humans and system components. New classes of user interfaces are emerging that make use of several sensors and are able to adapt their physical properties to the current situational context of users.

A key aspect of human-computer confluence is its potential for transforming human experience in the sense of bending, breaking and blending the barriers between the real, the virtual and the augmented, to allow users to experience their body and their world in new ways. Research on Presence, Embodiment and Brain-Computer Interface is already exploring these boundaries and asking questions such as: Can we seamlessly move between the virtual and the real? Can we assimilate fundamentally new senses through confluence?

The aim of this book is to explore the boundaries and intersections of the multidisciplinary field of HCC and discuss its potential applications in different domains, including healthcare, education, training and even arts.

May 26, 2016

In 1999, Joseph Pine and James Gilmore wrote a seminal book titled “The Experience Economy” (Harvard Business School Press, Boston, MA) that theorized the shift from a service-based economy to an experience-based economy.

According to these authors, in the new experience economy the goal of the purchase is no longer to own a product (be it a good or service), but to use it in order to enjoy a compelling experience. An experience, thus, is a whole-new type of offer: in contrast to commodities, goods and services, it is designed to be as personal and memorable as possible. Just as in a theatrical representation, companies stage meaningful events to engage customers in a memorable and personal way, by offering activities that provide engaging and rewarding experiences.

Indeed, if one looks back at the past ten years, the concept of experience has become more central to several fields, including tourism, architecture, and – perhaps more relevant for this column – to human-computer interaction, with the rise of “User Experience” (UX).

The concept of UX was introduced by Donald Norman in a 1995 article published on the CHI proceedings (D. Norman, J. Miller, A. Henderson: What You See, Some of What's in the Future, And How We Go About Doing It: HI at Apple Computer. Proceedings of CHI 1995, Denver, Colorado, USA). Norman argued that focusing exclusively on usability attribute (i.e. easy of use, efficacy, effectiveness) when designing an interactive product is not enough; one should take into account the whole experience of the user with the system, including users’ emotional and contextual needs. Since then, the UX concept has assumed an increasing importance in HCI. As McCarthy and Wright emphasized in their book “Technology as Experience” (MIT Press, 2004):

“In order to do justice to the wide range of influences that technology has in our lives, we should try to interpret the relationship between people and technology in terms of the felt life and the felt or emotional quality of action and interaction.” (p. 12).

However, according to Pine and Gilmore experience may not be the last step of what they call as “Progression of Economic Value”. They speculated further into the future, by identifying the “Transformation Economy” as the likely next phase. In their view, while experiences are essentially memorable events which stimulate the sensorial and emotional levels, transformations go much further in that they are the result of a series of experiences staged by companies to guide customers learning, taking action and eventually achieving their aspirations and goals.

In Pine and Gilmore terms, an aspirant is the individual who seeks advice for personal change (i.e. a better figure, a new career, and so forth), while the provider of this change (a dietist, a university) is an elictor. The elictor guide the aspirant through a series of experiences which are designed with certain purpose and goals. According to Pine and Gilmore, the main difference between an experience and a transformation is that the latter occurs when an experience is customized:

“When you customize an experience to make it just right for an individual - providing exactly what he needs right now - you cannot help changing that individual. When you customize an experience, you automatically turn it into a transformation, which companies create on top of experiences (recall that phrase: “a life-transforming experience”), just as they create experiences on top of services and so forth” (p. 244).

A further key difference between experiences and transformations concerns their effects: because an experience is inherently personal, no two people can have the same one. Likewise, no individual can undergo the same transformation twice: the second time it’s attempted, the individual would no longer be the same person (p. 254-255).

But what will be the impact of this upcoming, “transformation economy” on how people relate with technology? If in the experience economy the buzzword is “User Experience”, in the next stage the new buzzword might be “User Transformation”.

Indeed, we can see some initial signs of this shift. For example, FitBit and similar self-tracking gadgets are starting to offer personalized advices to foster enduring changes in users’ lifestyle; another example is from the fields of ambient intelligence and domotics, where there is an increasing focus towards designing systems that are able to learn from the user’s behaviour (i.e. by tracking the movement of an elderly in his home) to provide context-aware adaptive services (i.e. sending an alert when the user is at risk of falling).

But likely, the most important ICT step towards the transformation economy could take place with the introduction of next-generation immersive virtual reality systems. Since these new systems are based on mobile devices (an example is the recent partnership between Oculus and Samsung), they are able to deliver VR experiences that incorporate information on the external/internal context of the user (i.e. time, location, temperature, mood etc) by using the sensors incapsulated in the mobile phone.

By personalizing the immersive experience with context-based information, it might be possibile to induce higher levels of involvement and presence in the virtual environment. In case of cyber-therapeutic applications, this could translate into the development of more effective, transformative virtual healing experiences.

Furthermore, the emergence of "symbiotic technologies", such as neuroprosthetic devices and neuro-biofeedback, is enabling a direct connection between the computer and the brain. Increasingly, these neural interfaces are moving from the biomedical domain to become consumer products. But unlike existing digital experiential products, symbiotic technologies have the potential to transform more radically basic human experiences.

Brain-computer interfaces, immersive virtual reality and augmented reality and their various combinations will allow users to create “personalized alterations” of experience. Just as nowadays we can download and install a number of “plug-ins”, i.e. apps to personalize our experience with hardware and software products, so very soon we may download and install new “extensions of the self”, or “experiential plug-ins” which will provide us with a number of options for altering/replacing/simulating our sensorial, emotional and cognitive processes.

Such mediated recombinations of human experience will result from of the application of existing neuro-technologies in completely new domains. Although virtual reality and brain-computer interface were originally developed for applications in specific domains (i.e. military simulations, neurorehabilitation, etc), today the use of these technologies has been extended to other fields of application, ranging from entertainment to education.

In the field of biology, Stephen Jay Gould and Elizabeth Vrba (Paleobiology, 8, 4-15, 1982) have defined “exaptation” the process in which a feature acquires a function that was not acquired through natural selection. Likewise, the exaptation of neurotechnologies to the digital consumer market may lead to the rise of a novel “neuro-experience economy”, in which technology-mediated transformation of experience is the main product.

Just as a Genetically-Modified Organism (GMO) is an organism whose genetic material is altered using genetic-engineering techniques, so we could define aTechnologically-Modified Experience (ETM) a re-engineered experience resulting from the artificial manipulation of neurobiological bases of sensorial, affective, and cognitive processes.

Clearly, the emergence of the transformative neuro-experience economy will not happen in weeks or months but rather in years. It will take some time before people will find brain-computer devices on the shelves of electronic stores: most of these tools are still in the pre-commercial phase at best, and some are found only in laboratories.

Nevertheless, the mere possibility that such scenario will sooner or later come to pass, raises important questions that should be addressed before symbiotic technologies will enter our lives: does technological alteration of human experience threaten the autonomy of individuals, or the authenticity of their lives? How can we help individuals decide which transformations are good or bad for them?

Answering these important issues will require the collaboration of many disciplines, including philosophy, computer ethics and, of course, cyberpsychology.

Abstract. Neurofeedback is a promising approach for non-invasive modulation of human brain activity with applications for treatment of mental disorders and enhancement of brain performance. Neurofeedback techniques are commonly based on either electroencephalography (EEG) or real-time functional magnetic resonance imaging (rtfMRI). Advances in simultaneous EEG-fMRI have made it possible to combine the two approaches. Here we report the first implementation of simultaneous multimodal rtfMRI and EEG neurofeedback (rtfMRI-EEG-nf). It is based on a novel system for real-time integration of simultaneous rtfMRI and EEG data streams. We applied the rtfMRI-EEG-nf to training of emotional self-regulation in healthy subjects performing a positive emotion induction task based on retrieval of happy autobiographical memories. The participants were able to simultaneously regulate their BOLD fMRI activation in the left amygdala and frontal EEG power asymmetry in the high-beta band using the rtfMRI-EEG-nf. Our proof-of-concept results demonstrate the feasibility of simultaneous self-regulation of both hemodynamic (rtfMRI) and electrophysiological (EEG) activities of the human brain. They suggest potential applications of rtfMRI-EEG-nf in the development of novel cognitive neuroscience research paradigms and enhanced cognitive therapeutic approaches for major neuropsychiatric disorders, particularly depression.

Abstract. BACKGROUND: The day-to-day causes of stress are called daily hassles. Daily hassles are correlated with ill health. Biofeedback (BF) is one of the tools used for acquiring stress-coping skills. However, the anatomical correlates of the effects of BF with long training periods remain unclear. In this study, we aimed to investigate this. METHODS: PARTICIPANTS WERE ASSIGNED RANDOMLY TO TWO GROUPS: the intervention group and the control group. Participants in the intervention group performed a biofeedback training (BFT) task (a combination task for heart rate and cerebral blood flow control) every day, for about 5 min once a day. The study outcomes included MRI, psychological tests (e.g., Positive and Negative Affect Schedule, Center for Epidemiologic Studies Depression Scale, and Brief Job Stress Questionnaire), and a stress marker (salivary cortisol levels) before (day 0) and after (day 28) the intervention. RESULTS: We observed significant improvements in the psychological test scores and salivary cortisol levels in the intervention group compared to the control group. Furthermore, voxel-based morphometric analysis revealed that compared to the control group, the intervention group had significantly increased regional gray matter (GM) volume in the right lateral orbitofrontal cortex, which is an anatomical cluster that includes mainly the left hippocampus, and the left subgenual anterior cingulate cortex. The GM regions are associated with the stress response, and, in general, these regions seem to be the most sensitive to the detrimental effects of stress. CONCLUSIONS: Our findings suggest that our BFT is effective against the GM structures vulnerable to stress.

Abstract. Although the neurofeedback of real-time fMRI can reportedly enable people to gain control of the activity in the premotor cortex (PMA) during motor imagery, it is unclear how the neurofeedback training of PMA affect the motor network engaged in the motor execution (ME) and imagery (MI) task. In this study, we investigated the changes in the motor network engaged in both ME and MI task induced by real-time neurofeedback training of the right PMA. The neurofeedback training induced changes in activity of the ME-related motor network as well as alterations in the functional connectivity of both the ME-related and MI-related motor networks. Especially, the percent signal change of the right PMA in the last training run was found to be significantly correlated with the connectivity between the right PMA and the left posterior parietal lobe (PPL) during the pre-training MI run, post-training MI run and the last training run. Moreover, the increase in the tapping frequency was significantly correlated with the increase of connectivity between the right cerebellum and the primary motor area / primary sensory area (M1/S1) of the ME-related motor network after neurofeedback training. These findings show the importance of the connectivity between the right PMA and left PPL of the MI network for the up-regulation of the right PMA as well as the critical role of connectivity between the right cerebellum and M1/S1 of the ME network in improving the behavioral performance.

Jul 29, 2014

Abstract. PURPOSE OF REVIEW: The aim of this review is to provide a critical overview of recent research in the field of neuroscientific and clinical application of real-time functional MRI neurofeedback (rtfMRI-nf). RECENT FINDINGS: RtfMRI-nf allows self-regulating activity in circumscribed brain areas and brain systems. Furthermore, the learned regulation of brain activity has an influence on specific behaviors organized by the regulated brain regions. Patients with mental disorders show abnormal activity in certain regions, and simultaneous control of these regions using rtfMRI-nf may affect the symptoms of related behavioral disorders. SUMMARY: The promising results in clinical application indicate that rtfMRI-nf and other metabolic neurofeedback, such as near-infrared spectroscopy, might become a potential therapeutic tool. Further research is still required to examine whether rtfMRI-nf is a useful tool for psychiatry because there is still lack of knowledge about the neural function of certain brain systems and about neuronal markers for specific mental illnesses.

Apr 06, 2014

Researchers have developed a new way to explore the human brain in virtual reality. The system, called Glass Brain, which is developed by Philip Rosedale, creator of the famous game Second Life, and Adam Gazzaley, a neuroscientist at the University of California San Francisco, combines brain scanning, brain recording and virtual reality to allow a user to journey through a person’s brain in real-time.

Abstract. Biofeedback of heart rate variability (HRV) was applied to patients with diabetic polyneuropathy using a new mobile device allowing regularly scheduled self-measurements without the need of visits to a special autonomic laboratory. Prolonged generation of data over an eight-week period facilitated more precise investigation of cardiac autonomic function and assessment of positive and negative trends of HRV parameters over time. Statistical regression analyses revealed significant trends in 11 of 17 patients, while no significant differences were observed when comparing autonomic screening by short-term HRV and respiratory sinus arrhythmia at baseline and after the 8 weeks training period. Four patients showed positive trends of HRV parameters despite the expected progression of cardiac autonomic dysfunction over time. Patient compliance was above 50% in all but two patients. The results of this preliminary study indicate a good practicality of the handheld device and suggest a potential positive effect on cardiac autonomic neuropathy in patients with type 2 diabetes.

Dec 24, 2013

Among the clinical applications of neurofeedback, most research has been conducted in ADHD. As an introduction a short overview of the general history of neurofeedback will be given, while the main part of the paper deals with a review of the current state of neurofeedback in ADHD. A meta-analysis on neurofeedback from 2009 found large effect sizes for inattention and impulsivity and medium effects sizes for hyperactivity. Since 2009 several new studies, including 4 placebo-controlled studies, have been published. These latest studies are reviewed and discussed in more detail. The review focuses on studies employing 1) semi-active, 2) active, and 3) placebo-control groups. The assessment of specificity of neurofeedback treatment in ADHD is discussed and it is concluded that standard protocols such as theta/beta, SMR and slow cortical potentials neurofeedback are well investigated and have demonstrated specificity. The paper ends with an outlook on future questions and tasks. It is concluded that future controlled clinical trials should, in a next step, focus on such known protocols, and be designed along the lines of learning theory.

Abstract. The demonstration that humans can learn to modulate their own brain activity based on feedback of neurophysiological signals opened up exciting opportunities for fundamental and applied neuroscience. Although EEG-based neurofeedback has been long employed both in experimental and clinical investigation, functional MRI (fMRI)-based neurofeedback emerged as a promising method, given its superior spatial resolution and ability to gauge deep cortical and subcortical brain regions. In combination with improved computational approaches, such as pattern recognition analysis (e.g., Support Vector Machines, SVM), fMRI neurofeedback and brain decoding represent key innovations in the field of neuromodulation and functional plasticity. Expansion in this field and its applications critically depend on the existence of freely available, integrated and user-friendly tools for the neuroimaging research community. Here, we introduce FRIEND, a graphic-oriented user-friendly interface package for fMRI neurofeedback and real-time multivoxel pattern decoding. The package integrates routines for image preprocessing in real-time, ROI-based feedback (single-ROI BOLD level and functional connectivity) and brain decoding-based feedback using SVM. FRIEND delivers an intuitive graphic interface with flexible processing pipelines involving optimized procedures embedding widely validated packages, such as FSL and libSVM. In addition, a user-defined visual neurofeedback module allows users to easily design and run fMRI neurofeedback experiments using ROI-based or multivariate classification approaches. FRIEND is open-source and free for non-commercial use. Processing tutorials and extensive documentation are available.

Nov 16, 2013

Neurofeedback training aimed to improve focused attention and alertness in children with ADHD: a study of relative power of EEG rhythms using custom-made software application.

Clin EEG Neurosci. 2013 Jul;44(3):193-202

Authors: Hillard B, El-Baz AS, Sears L, Tasman A, Sokhadze EM

Abstract. Neurofeedback is a nonpharmacological treatment for attention-deficit hyperactivity disorder (ADHD). We propose that operant conditioning of electroencephalogram (EEG) in neurofeedback training aimed to mitigate inattention and low arousal in ADHD, will be accompanied by changes in EEG bands' relative power. Patients were 18 children diagnosed with ADHD. The neurofeedback protocol ("Focus/Alertness" by Peak Achievement Trainer) has a focused attention and alertness training mode. The neurofeedback protocol provides one for Focus and one for Alertness. This does not allow for collecting information regarding changes in specific EEG bands (delta, theta, alpha, low and high beta, and gamma) power within the 2 to 45 Hz range. Quantitative EEG analysis was completed on each of twelve 25-minute-long sessions using a custom-made MatLab application to determine the relative power of each of the aforementioned EEG bands throughout each session, and from the first session to the last session. Additional statistical analysis determined significant changes in relative power within sessions (from minute 1 to minute 25) and between sessions (from session 1 to session 12). Analysis was of relative power of theta, alpha, low and high beta, theta/alpha, theta/beta, and theta/low beta and theta/high beta ratios. Additional secondary measures of patients' post-neurofeedback outcomes were assessed, using an audiovisual selective attention test (IVA + Plus) and behavioral evaluation scores from the Aberrant Behavior Checklist. Analysis of data computed in the MatLab application, determined that theta/low beta and theta/alpha ratios decreased significantly from session 1 to session 12, and from minute 1 to minute 25 within sessions. The findings regarding EEG changes resulting from brain wave self-regulation training, along with behavioral evaluations, will help elucidate neural mechanisms of neurofeedback aimed to improve focused attention and alertness in ADHD.

It has been demonstrated that visual inputs can modulate pain. However, the influence of skin color on pain perception is unknown. Red skin is associated to inflamed, hot and more sensitive skin, while blue is associated to cyanotic, cold skin. We aimed to test whether the color of the skin would alter the heat pain threshold. To this end, we used an immersive virtual environment where we induced embodiment of a virtual arm that was co-located with the real one and seen from a first-person perspective. Virtual reality allowed us to dynamically modify the color of the skin of the virtual arm. In order to test pain threshold, increasing ramps of heat stimulation applied on the participants' arm were delivered concomitantly with the gradual intensification of different colors on the embodied avatar's arm. We found that a reddened arm significantly decreased the pain threshold compared with normal and bluish skin. This effect was specific when red was seen on the arm, while seeing red in a spot outside the arm did not decrease pain threshold. These results demonstrate an influence of skin color on pain perception. This top-down modulation of pain through visual input suggests a potential use of embodied virtual bodies for pain therapy.

Physiological data in its different dimensions, either bioelectrical, biomechanical, biochemical or biophysical, and collected through specialized biomedical devices, video and image capture or other sources, is opening new boundaries in the field of human-computer interaction into what can be defined as Physiological Computing. PhyCS is the annual meeting of the physiological interaction and computing community, and serves as the main international forum for engineers, computer scientists and health professionals, interested in outstanding research and development that bridges the gap between physiological data handling and human-computer interaction.

Jul 23, 2013

NeuroMetrix of out of Waltham, MA received FDA clearance for its SENSUS Pain Management System to be used by patients during sleep. This is the first transcutaneous electrical nerve stimulation system to receive a sleep indication from the FDA for pain control.

The device is designed for use by diabetics and others with chronic pain in the legs and feet. It’s worn around one or both legs and delivers an electrical current to disrupt pain signals being sent up to the brain.

Abstract. The present study described a new type of multi-person neurofeedback with the neural synchronization between two participants as the direct regulating target, termed as "cross-brain neurofeedback." As a first step to implement this concept, an experimental platform was built on the basis of functional near-infrared spectroscopy, and was validated with a two-person neurofeedback experiment. This novel concept as well as the experimental platform established a framework for investigation of the relationship between multiple participants' cross-brain neural synchronization and their social behaviors, which could provide new insight into the neural substrate of human social interactions.

Abstract. Studies on the potential benefits of conveying biofeedback stimulus using a musical signal have appeared in recent years with the intent of harnessing the strong effects that music listening may have on subjects. While results are encouraging, the fundamental question has yet to be addressed, of how combined music and biofeedback compares to the already established use of either of these elements separately. This experiment, involving young adults (N=24), compared the effectiveness at modulating participants' states of physiological arousal of each of the following conditions: A) listening to pre-recorded music, B) sonification biofeedback of the heart rate, and C) an algorithmically modulated musical feedback signal conveying the subject's heart rate. Our hypothesis was that each of the conditions (A), (B) and (C) would differ from the other two in the extent to which it enables participants to increase and decrease their state of physiological arousal, with (C) being more effective than (B), and both more than (A). Several physiological measures and qualitative responses were recorded and analyzed. Results show that using musical biofeedback allowed participants to modulate their state of physiological arousal at least equally well as sonification biofeedback, and much better than just listening to music, as reflected in their heart rate measurements, controlling for respiration-rate. Our findings indicate that the known effects of music in modulating arousal can therefore be beneficially harnessed when designing a biofeedback protocol.

Application of alpha/theta neurofeedback and heart rate variability training to young contemporary dancers: State anxiety and creativity.

Int J Psychophysiol. 2013 May 15;

Authors: Gruzelier JH, Thompson T, Redding E, Brandt R, Steffert T

Abstract. As one in a series on the impact of EEG-neurofeedback in the performing arts, we set out to replicate a previous dance study in which alpha/theta (A/T) neurofeedback and heart rate variability (HRV) biofeedback enhanced performance in competitive ballroom dancers compared with controls. First year contemporary dance conservatoire students were randomised to the same two psychophysiological interventions or a choreology instruction comparison group or a no-training control group. While there was demonstrable neurofeedback learning, there was no impact of the three interventions on dance performance as assessed by four experts. However, HRV training reduced anxiety and the reduction correlated with improved technique and artistry in performance; the anxiety scale items focussed on autonomic functions, especially cardiovascular activity. In line with the putative impact of hypnogogic training on creativity A/T training increased cognitive creativity with the test of unusual uses, but not insight problems. Methodological and theoretical implications are considered.

Abstract. Accumulating evidence indicates that motor imagery and motor execution share common neural networks. Accordingly, mental practices in the form of motor imagery have been implemented in rehabilitation regimes of stroke patients with favorable results. Because direct monitoring of motor imagery is difficult, feedback of cortical activities related to motor imagery (neurofeedback) could help to enhance efficacy of mental practice with motor imagery. To determine the feasibility and efficacy of a real-time neurofeedback system mediated by near-infrared spectroscopy (NIRS), two separate experiments were performed. Experiment 1 was used in five subjects to evaluate whether real-time cortical oxygenated hemoglobin signal feedback during a motor execution task correlated with reference hemoglobin signals computed off-line. Results demonstrated that the NIRS-mediated neurofeedback system reliably detected oxygenated hemoglobin signal changes in real-time. In Experiment 2, 21 subjects performed motor imagery of finger movements with feedback from relevant cortical signals and irrelevant sham signals. Real neurofeedback induced significantly greater activation of the contralateral premotor cortex and greater self-assessment scores for kinesthetic motor imagery compared with sham feedback. These findings suggested the feasibility and potential effectiveness of a NIRS-mediated real-time neurofeedback system on performance of kinesthetic motor imagery. However, these results warrant further clinical trials to determine whether this system could enhance the effects of mental practice in stroke patients.

Mar 31, 2012

Neurofeedback for insomnia: a pilot study of Z-score SMR and individualized protocols.

Appl Psychophysiol Biofeedback. 2011 Dec;36(4):251-64

Authors: Hammer BU, Colbert AP, Brown KA, Ilioi EC

Abstract. Insomnia is an epidemic in the US. Neurofeedback (NFB) is a little used, psychophysiological treatment with demonstrated usefulness for treating insomnia. Our objective was to assess whether two distinct Z-Score NFB protocols, a modified sensorimotor (SMR) protocol and a sequential, quantitative EEG (sQEEG)-guided, individually designed (IND) protocol, would alleviate sleep and associated daytime dysfunctions of participants with insomnia. Both protocols used instantaneous Z scores to determine reward condition administered when awake. Twelve adults with insomnia, free of other mental and uncontrolled physical illnesses, were randomly assigned to the SMR or IND group. Eight completed this randomized, parallel group, single-blind study. Both groups received fifteen 20-min sessions of Z-Score NFB. Pre-post assessments included sQEEG, mental health, quality of life, and insomnia status. ANOVA yielded significant post-treatment improvement for the combined group on all primary insomnia scores: Insomnia Severity Index (ISI p<.005), Pittsburgh Sleep Quality Inventory (PSQI p<.0001), PSQI Sleep Efficiency (p<.007), and Quality of Life Inventory (p<.02). Binomial tests of baseline EEGs indicated a significant proportion of excessively high levels of Delta and Beta power (p<.001) which were lowered post-treatment (paired z-tests p<.001). Baseline EEGs showed excessive sleepiness and hyperarousal, which improved post-treatment. Both Z-Score NFB groups improved in sleep and daytime functioning. Post-treatment, all participants were normal sleepers. Because there were no significant differences in the findings between the two groups, our future large scale studies will utilize the less burdensome to administer Z-Score SMR protocol.