NIRS-based neurofeedback learning systems for controlling activity of the prefrontal cortex

Subject-independent decoding of affective states using functional near-infrared spectroscopy

Affective decoding is the inference of human emotional states using brain signal measurements. This approach is crucial to develop new therapeutic approaches for psychiatric rehabilitation, such as affective neurofeedback protocols. To reduce the training duration and optimize the clinical outputs, an ideal clinical neurofeedback could be trained using data from an independent group of volunteers before being used by new patients. Here, we investigated if this subject-independent design of affective decoding can be achieved using functional near-infrared spectroscopy (fNIRS) signals from frontal and occipital areas. For this purpose, a linear discriminant analysis classifier was first trained in a dataset (49 participants, 24.65±3.23 years) and then tested in a completely independent one (20 participants, 24.00±3.92 years). Significant balanced accuracies between classes were found for positive vs. negative (64.50 ± 12.03%, p<0.01) and negative vs. neutral (68.25 ± 12.97%, p<0.01) affective states discrimination during a reactive block consisting in viewing affective-loaded images. For an active block, in which volunteers were instructed to recollect personal affective experiences, significant accuracy was found for positive vs. neutral affect classification (71.25 ± 18.02%, p<0.01). In this last case, only three fNIRS channels were enough to discriminate between neutral and positive affective states. Although more research is needed, for example focusing on better combinations of features and classifiers, our results highlight fNIRS as a possible technique for subject-independent affective decoding, reaching significant classification accuracies of emotional states using only a few but biologically relevant features.

Prefrontal Asymmetry BCI Neurofeedback Datasets

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Differential Subjective Experiences in Learners and Non-learners in Frontal Alpha Neurofeedback: Piloting a Mixed-Method Approach

In a neurofeedback paradigm, trainees learn to willfully control their brain dynamics. How this is realized remains an open question. We evaluate the hypothesis that learning success is associated with a specific phenomenology. To address this proposal, we combined quantitative and qualitative analyses of a short neurofeedback training (NFT) session during which participants enhanced mid-frontal alpha power and were then subsequently interviewed about their experiences. We analyzed the electrophysiological data to determine learning success and classify trainees as learners and non-learners. The subjective experiences differed between the two groups and are best described along a trying-sensing continuum, with non-learners engaging effortfully with the task (e.g., “I will it [the bar] to move”) whereas learners reported more sensing of their inner (e.g., “Something inside my stomach”) and outer environment (e.g., “I was aware of the sound of the beeps”). In the process of piloting this mixed-method approach, we developed a classification system for the verbal reports. This system provides an explicit analytic framework which might guide future studies that aim to investigate the association between subjective experiences and NFT protocols.

Predicting affective valence using cortical hemodynamic signals

Ascribing affective valence to stimuli or mental states is a fundamental property of human experiences. Recent neuroimaging meta-analyses favor the workspace hypothesis for the neural underpinning of valence, in which both positive and negative values are encoded by overlapping networks but are associated with different patterns of activity. In the present study, we further explored this framework using functional near-infrared spectroscopy (fNIRS) in conjunction with multivariate analyses. We monitored the fronto-temporal and occipital hemodynamic activity of 49 participants during the viewing of affective images (passive condition) and during the imagination of affectively loaded states (active condition). Multivariate decoding techniques were applied to determine whether affective valence is encoded in the cortical areas assessed. Prediction accuracies of 89.90 ± 13.84% and 85.41 ± 14.43% were observed for positive versus neutral comparisons, and of 91.53 ± 13.04% and 81.54 ± 16.05% for negative versus neutral comparisons (passive/active conditions, respectively). Our results are consistent with previous studies using other neuroimaging modalities that support the affective workspace hypothesis and the notion that valence is instantiated by the same network, regardless of whether the affective experience is passively or actively elicited.

BCI Control of Heuristic Search Algorithms

The ability to develop Brain-Computer Interfaces (BCI) to Intelligent Systems would offer new perspectives in terms of human supervision of complex Artificial Intelligence (AI) systems, as well as supporting new types of applications. In this article, we introduce a basic mechanism for the control of heuristic search through fNIRS-based BCI. The rationale is that heuristic search is not only a basic AI mechanism but also one still at the heart of many different AI systems. We investigate how users’ mental disposition can be harnessed to influence the performance of heuristic search algorithm through a mechanism of precision-complexity exchange. From a system perspective, we use weighted variants of the A* algorithm which have an ability to provide faster, albeit suboptimal solutions. We use recent results in affective BCI to capture a BCI signal, which is indicative of a compatible mental disposition in the user. It has been established that Prefrontal Cortex (PFC) asymmetry is strongly correlated to motivational dispositions and results anticipation, such as approach or even risk-taking, and that this asymmetry is amenable to Neurofeedback (NF) control. Since PFC asymmetry is accessible through fNIRS, we designed a BCI paradigm in which users vary their PFC asymmetry through NF during heuristic search tasks, resulting in faster solutions. This is achieved through mapping the PFC asymmetry value onto the dynamic weighting parameter of the weighted A* (WA*) algorithm. We illustrate this approach through two different experiments, one based on solving 8-puzzle configurations, and the other on path planning. In both experiments, subjects were able to speed up the computation of a solution through a reduction of search space in WA*. Our results establish the ability of subjects to intervene in heuristic search progression, with effects which are commensurate to their control of PFC asymmetry: this opens the way to new mechanisms for the implementation of hybrid cognitive systems.

Development of a neurofeedback protocol targeting the frontal pole using near-infrared spectroscopy

AIM

Neurofeedback has been studied with the aim of controlling cerebral activity. Near-infrared spectroscopy is a non-invasive neuroimaging technique used for measuring hemoglobin concentration changes in cortical surface areas with high temporal resolution. Thus, near-infrared spectroscopy may be useful for neurofeedback, which requires real-time feedback of repeated brain activation measurements. However, no study has specifically targeted neurofeedback, using near-infrared spectroscopy, in the frontal pole cortex.

METHODS

We developed an original near-infrared spectroscopy neurofeedback system targeting the frontal pole cortex. Over a single day of testing, each healthy participant (n = 24) received either correct or incorrect (Sham) feedback from near-infrared spectroscopy signals, based on a crossover design.

RESULTS

Under correct feedback conditions, significant activation was observed in the frontal pole cortex (P = 0.000073). Additionally, self-evaluation of control and metacognitive beliefs were associated with near-infrared spectroscopy signals (P = 0.006).

CONCLUSION

The neurofeedback system developed in this study might be useful for developing control of frontal pole cortex activation.

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface

Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.