What we can and cannot (yet) do with functional near infrared spectroscopy

Mechanisms underlying fNIRS-neurofeedback over the prefrontal cortex for participants with binge-eating disorder

OBJECTIVE

Despite the increasing popularity of neurofeedback (NF), aiming at voluntary modulation of dysfunctional prefrontal cortex (PFC) signals in the treatment of binge-eating disorder (BED) and/or overweight, mechanisms remain poorly understood.

METHODS

Based on a randomized-controlled trial offering 12 food-specific real-time functional near-infrared spectroscopy (rtfNIRS)-NF sessions to participants with BED (n = 22), this preregistered study examined (1) online regulation success as predictor for offline regulation success, defined by PFC signals during regulation versus watch, and subjective regulation success, and (2) changes in loss of control (LOC) eating after vs. before and across 12 rtfNIRS-NF-sessions.

RESULTS

Higher online regulation success expectedly predicted better subjective, but worse offline regulation success. LOC eating decreased after vs. before, but not over rtfNIRS-NF-sessions, and was not associated with subjective or offline regulation success.

CONCLUSIONS

The association between online and subjective regulation success confirmed the presumed mechanism of operant conditioning underlying rtfNIRS-NF-learning. The contrary association between online and offline regulation indicated differential PFC involvement upon subtraction of automatic food-specific responses from regulation signals for offline success. Decreased LOC eating after food-specific rtfNIRS-NF-sessions suggested the potential of NF in BED treatment.

SIGNIFICANCE

Results may guide the optimization of future NF studies in larger samples with BED.

Temel Duygusal Durumların Hemodinamik Karşılıklarının Taşınabilir bir İşlevsel Yakın Kızılaltı Spektroskopi Sistemi ile Tanımlanması

Objective: The aim of this study was to test the feasibility of a functional near infrared spectroscopy (fNIRS) system for quantification of the similarities and differences in the spatial localization of cerebral hemodynamic activation induced by visual presentation of neutral, negative and positive valence emotional stimuli. Method: 13 healthy subjects viewed neutral, pleasant and unpleasant pictures from the International Affective Picture System (IAPS) database in a block design experiment while the prefrontal cortical hemodynamic changes induced by emotional stimuli were continuously recorded with a 20 channel fNIRS system that covered the forehead region. Results: Negative valence pictures induced higher hemodynamic activity in right lateralized regions involving dorsolateral and oribtofrontal cortex when compared to neutral and positive valence stimuli (pFDR

Supervised Classification of Operator Functional State Based on Physiological Data: Application to Drones Swarm Piloting

To improve the safety and the performance of operators involved in risky and demanding missions (like drone operators), human-machine cooperation should be dynamically adapted, in terms of dialogue or function allocation. To support this reconfigurable cooperation, a crucial point is to assess online the operator’s ability to keep performing the mission. The article explores the concept of Operator Functional State (OFS), then it proposes to operationalize this concept (combining context and physiological indicators) on the specific activity of drone swarm monitoring, carried out by 22 participants on simulator SUSIE. With the aid of supervised learning methods (Support Vector Machine, k-Nearest Neighbors, and Random Forest), physiological and contextual are classified into three classes, corresponding to different levels of OFS. This classification would help for adapting the countermeasures to the situation faced by operators.

Future Applications of Real-World Neuroimaging to Clinical Psychology

Clinical neuroimaging has largely been limited to examining the neurophysiological outcomes of treatments for psychiatric conditions rather than the neurocognitive mechanisms by which these outcomes are brought about as a function of clinical strategies, and the cognitive neuroscientific research aiming to investigate these mechanisms in nonclinical and clinical populations has been ecologically challenged by the extent to which tasks represent and generalize to intervention strategies. However, recent technological and methodological advancements to neuroimaging techniques such as functional near-infrared spectroscopy and functional near-infrared spectroscopy-based hyperscanning provide novel opportunities to investigate the mechanisms of change in more naturalistic and interactive settings, representing a unique prospect for improving our understanding of the intra- and interbrain systems supporting the recogitation of dysfunctional cognitive operations.

Affective Neurofeedback Under Naturalistic Conditions: A Mini-Review of Current Achievements and Open Challenges

Affective neurofeedback training allows for the self-regulation of the putative circuits of emotion regulation. This approach has recently been studied as a possible additional treatment for psychiatric disorders, presenting positive effects in symptoms and behaviors. After neurofeedback training, a critical aspect is the transference of the learned self-regulation strategies to outside the laboratory and how to continue reinforcing these strategies in non-controlled environments. In this mini-review, we discuss the current achievements of affective neurofeedback under naturalistic setups. For this, we first provide a brief overview of the state-of-the-art for affective neurofeedback protocols. We then discuss virtual reality as a transitional step toward the final goal of "in-the-wild" protocols and current advances using mobile neurotechnology. Finally, we provide a discussion of open challenges for affective neurofeedback protocols in-the-wild, including topics such as convenience and reliability, environmental effects in attention and workload, among others.

Morality and management: an oxymoron? fNIRS and neuromanagement perspective explain us why things are not like this

The neuroscience interest for moral decision-making has recently increased. To investigate the processes underlying moral behavior, this research aimed to investigate neurophysiological and behavioral correlates of decision-making in moral contexts. Specifically, functional Near-infrared spectroscopy (fNIRS) allowed to record oxygenated (O2Hb) and deoxygenated (HHb) cerebral hemoglobin concentrations during different moral conditions (professional fit, company fit, social fit) and offers types (fair, unfair, neutral). Moreover, individuals' responses to offers types and reaction time (RTs) were considered. Specifically, from hemodynamic results emerged a difference in O2Hb and HHb activity according to moral conditions and offers types in different brain regions. In particular, O2Hb increase and a HHb decrease were observed in ventromedial and dorsolateral prefrontal cortex (VMPFC, DLPFC) for fair offers in professional fit condition and in superior temporal sulcus (STS) for unfair offers in social fit condition. Moreover, an increase of left O2Hb activity in professional fit condition and in right VMPFC for unfair offers in company fit condition was observed. In addition, from behavioral results, an RTs increase in company and social fit condition for fair and unfair offers emerged. This study, therefore, shows the behavioral and neurophysiological correlates of moral decision-making that guide moral behavior in different context, such as company one.

Integration of Spatial Information Increases Reproducibility in Functional Near-Infrared Spectroscopy

As functional near-infrared spectroscopy (fNIRS) is developed as a neuroimaging technique and becomes an option to study a variety of populations and tasks, the reproducibility of the fNIRS signal is still subject of debate. By performing test-retest protocols over different functional tasks, several studies agree that the fNIRS signal is reproducible over group analysis, but the inter-subject and within-subject reproducibility is poor. The high variability at the first statistical level is often attributed to global systemic physiology. In the present work, we revisited the reproducibility of the fNIRS signal during a finger-tapping task across multiple sessions on the same and different days. We expanded on previous studies by hypothesizing that the lack of spatial information of the optodes contributes to the low reproducibility in fNIRS, and we incorporated a real-time neuronavigation protocol to provide accurate cortical localization of the optodes. Our proposed approach was validated in 10 healthy volunteers, and our results suggest that the addition of neuronavigation can increase the within-subject reproducibility of the fNIRS data, particularly in the region of interest. Unlike traditional approaches to positioning the optodes, in which low intra-subject reproducibility has been found, we were able to obtain consistent and robust activation of the contralateral primary motor cortex at the intra-subject level using a neuronavigation protocol. Overall, our findings support the hypothesis that at least part of the variability in fNIRS cannot be only attributed to global systemic physiology. The use of neuronavigation to guide probe positioning, as proposed in this work, has impacts to longitudinal protocols performed with fNIRS.

Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface

Cognitive workload is one of the widely invoked human factors in the areas of human-machine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Real-time four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.

Human Discrimination and Categorization of Emotions in Voices: A Functional Near-Infrared Spectroscopy (fNIRS) Study

Functional Near-Infrared spectroscopy (fNIRS) is a neuroimaging tool that has been recently used in a variety of cognitive paradigms. Yet, it remains unclear whether fNIRS is suitable to study complex cognitive processes such as categorization or discrimination. Previously, functional imaging has suggested a role of both inferior frontal cortices in attentive decoding and cognitive evaluation of emotional cues in human vocalizations. Here, we extended paradigms used in functional magnetic resonance imaging (fMRI) to investigate the suitability of fNIRS to study frontal lateralization of human emotion vocalization processing during explicit and implicit categorization and discrimination using mini-blocks and event-related stimuli. Participants heard speech-like but semantically meaningless pseudowords spoken in various tones and evaluated them based on their emotional or linguistic content. Behaviorally, participants were faster to discriminate than to categorize; and processed the linguistic faster than the emotional content of stimuli. Interactions between condition (emotion/word), task (discrimination/categorization) and emotion content (anger, fear, neutral) influenced accuracy and reaction time. At the brain level, we found a modulation of the Oxy-Hb changes in IFG depending on condition, task, emotion and hemisphere (right or left), highlighting the involvement of the right hemisphere to process fear stimuli, and of both hemispheres to treat anger stimuli. Our results show that fNIRS is suitable to study vocal emotion evaluation, fostering its application to complex cognitive paradigms.

Decoding Mental Workload in Virtual Environments: A fNIRS Study using an Immersive n-back Task

Virtual Reality (VR) has emerged as a novel paradigm for immersive applications in training, entertainment, rehabilitation, and other domains. In this paper, we investigate the automatic classification of mental workload from brain activity measured through functional near-infrared spectroscopy (fNIRS) in VR. We present results from a study which implements the established n-back task in an immersive visual scene, including physical interaction. Our results show that user workload can be detected from fNIRS signals in immersive VR tasks both person-dependently and -adaptively.

Functional near-infrared spectroscopy for speech protocols: characterization of motion artifacts and guidelines for improving data analysis

Monitoring speech tasks with functional near-infrared spectroscopy (fNIRS) enables investigation of speech production mechanisms and informs treatment strategies for speech-related disorders such as stuttering. Unfortunately, due to movement of the temporalis muscle, speech production can induce relative movement between probe optodes and skin. These movements generate motion artifacts during speech tasks. In practice, spurious hemodynamic responses in functional activation signals arise from lack of information about the consequences of speech-related motion artifacts, as well as from lack of standardized processing procedures for fNIRS signals during speech tasks. To this end, we characterize the effects of speech production on fNIRS signals, and we introduce a systematic analysis to ameliorate motion artifacts. The study measured 50 healthy subjects performing jaw movement (JM) tasks and found that JM produces two different patterns of motion artifacts in fNIRS. To remove these unwanted contributions, we validate a hybrid motion-correction algorithm based sequentially on spline interpolation and then wavelet filtering. We compared performance of the hybrid algorithm with standard algorithms based on spline interpolation only and wavelet decomposition only. The hybrid algorithm corrected 94% of the artifacts produced by JM, and it did not lead to spurious responses in the data. We also validated the hybrid algorithm during a reading task performed under two different conditions: reading aloud and reading silently. For both conditions, we observed significant cortical activation in brain regions related to reading. Moreover, when comparing the two conditions, good agreement of spatial and temporal activation patterns was found only when data were analyzed using the hybrid approach. Overall, the study demonstrates a standardized processing scheme for fNIRS data during speech protocols. The scheme decreases spurious responses and intersubject variability due to motion artifacts.

Characterizing cortical hemodynamic changes during climbing and its relation to climbing expertise

Bouldering is a special form of climbing without rope that requires coordinated whole-body movements. While physical performance parameters such as condition have been well studied, the knowledge on neural activity during climbing still remains sparse. Functional near-infrared spectroscopy (fNIRS) allows to measure brain activation while performing sportive actions due to its relative robustness against motion artefacts. In the current study, hemodynamic response alterations of 13 advanced climbers were investigated during boulder performance using fNIRS measurements. Simple and moderate climbing routes were compared regarding their level of cortical activation mainly in the sensorimotor area. Our results show that repetitively climbing a set of boulders activates almost all areas of the sensorimotor system including the bilateral premotor and supplementary motor cortex, bilateral primary motor cortex as well as the bilateral gyrus supramarginalis and somatosensory cortex. This result was found in both simple and moderate climbing routes with no effect of task complexity on the level of cortical activity. Correlation analysis (uncorrected for multiple comparisons) revealed a negative association between the level of expertise and the hemodynamic response in the supplementary-motor region, suggesting that gaining expertise in climbing is associated with a decrease in secondary motor areas, which is an indicator of motor automaticity. In summary, the present study provides first proof of concept that fNIRS is capable of assessing hemodynamic response alterations within the human motor system during the execution of complex whole-body climbing movements.

Functional Spectroscopy Mapping of Pain Processing Cortical Areas During Non-painful Peripheral Electrical Stimulation of the Accessory Spinal Nerve

Peripheral electrical stimulation (PES), which encompasses several techniques with heterogeneous physiological responses, has shown in some cases remarkable outcomes for pain treatment and clinical rehabilitation. However, results are still mixed, mainly because there is a lack of understanding regarding its neural mechanisms of action. In this study, we aimed to assess its effects by measuring cortical activation as indexed by functional near infrared spectroscopy (fNIRS). fNIRS is a functional optical imaging method to evaluate hemodynamic changes in oxygenated (HbO) and de-oxygenated (HbR) blood hemoglobin concentrations in cortical capillary networks that can be related to cortical activity. We hypothesized that non-painful PES of accessory spinal nerve (ASN) can promote cortical activation of sensorimotor cortex (SMC) and dorsolateral prefrontal cortex (DLPFC) pain processing cortical areas. Fifteen healthy volunteers received both active and sham ASN electrical stimulation in a crossover study. The hemodynamic cortical response to unilateral right ASN burst electrical stimulation with 10 Hz was measured by a 40-channel fNIRS system. The effect of ASN electrical stimulation over HbO concentration in cortical areas of interest (CAI) was observed through the activation of right-DLPFC (p = 0.025) and left-SMC (p = 0.042) in the active group but not in sham group. Regarding left-DLPFC (p = 0.610) and right-SMC (p = 0.174) there was no statistical difference between groups. As in non-invasive brain stimulation (NIBS) top-down modulation, bottom-up electrical stimulation to the ASN seems to activate the same critical cortical areas on pain pathways related to sensory-discriminative and affective-motivational pain dimensions. These results provide additional mechanistic evidence to develop and optimize the use of peripheral nerve electrical stimulation as a neuromodulatory tool (NCT 03295370— www.clinicaltrials.gov).

Measuring Cortical Activity During Auditory Processing with Functional Near-Infrared Spectroscopy

Functional near-infrared spectroscopy (fNIRS) is an optical, non-invasive neuroimaging technique that investigates human brain activity by calculating concentrations of oxy- and deoxyhemoglobin. The aim of this publication is to review the current state of the art as to how fNIRS has been used to study auditory function. We address temporal and spatial characteristics of the hemodynamic response to auditory stimulation as well as experimental factors that affect fNIRS data such as acoustic and stimulus-driven effects. The rising importance that fNIRS is generating in auditory neuroscience underlines the strong potential of the technology, and it seems likely that fNIRS will become a useful clinical tool.

How to Find Effects of Stimulus Processing on Event Related Brain Potentials of Close Others when Hyperscanning Partners

The partners of each pair must be able to pass the McGill Friendship Questionnaire without communicating. Each partner is then seated in front of a screen in one of two adjacent rooms. These rooms are separated by a glass window through which participants communicate to maintain feelings of togetherness while being fitted with the EEG cap. After checking for adequate EEG signals, the glass is covered by a curtain to prevent visual communication. Then, partners must be silent but are instructed to try to feel in the presence of their partner during the entire experiment. Just before it starts, participants are told that each of them will be presented with one image at a time and that these images will occur at the same time for both of them on their own screen. They are also instructed that, for each trial, the simultaneous images will always be different. However, unbeknownst to them, trials are randomized: only half of them are consistent with this instruction and actually include two different images. These trials form the DSC, that is, the different-stimuli condition. The other half of the trials are inconsistent with the instruction. They include two identical images and form the ISC (identical-stimuli condition). After the experiment, participants are sorted into two groups: those who reported that they felt in the presence of their partner during the majority of the trials and those who reported they did not. The impact of the stimulus processing of the partner is found by subtracting the mean voltages of the ERPs of the ISC (inconsistent with the instructions) from the ERPs of the DSC (consistent with the instructions) in at least two time windows (TWs): firstly, in the 75 to 150 ms TW, where the absolute values of these subtractions are greater, especially at right frontal sites, in those who felt in the presence of their partner than in those who did not; secondly, in the LPP time window (i.e., from 650 to 950 ms post onset), where ERPs are significantly less positive in the DSC than in the ISC in those in whom the raw results of the early (75-150ms) subtractions are negative.

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations.

A functional near-infrared spectroscopic investigation of speech production during reading

This study was designed to test the extent to which speaking processes related to articulation and voicing influence Functional Near Infrared Spectroscopy (fNIRS) measures of cortical hemodynamics and functional connectivity. Participants read passages in three conditions (oral reading, silent mouthing, and silent reading) while undergoing fNIRS imaging. Area under the curve (AUC) analyses of the oxygenated and deoxygenated hemodynamic response function concentration values were compared for each task across five regions of interest. There were significant region main effects for both oxy and deoxy AUC analyses, and a significant region × task interaction for deoxy AUC favoring the oral reading condition over the silent reading condition for two nonmotor regions. Assessment of functional connectivity using Granger Causality revealed stronger networks between motor areas during oral reading and stronger networks between language areas during silent reading. There was no evidence that the hemodynamic flow from motor areas during oral reading compromised measures of language-related neural activity in nonmotor areas. However, speech movements had small, but measurable effects on fNIRS measures of neural connections between motor and nonmotor brain areas across the perisylvian region, even after wavelet filtering. Therefore, researchers studying speech processes with fNIRS should use wavelet filtering during preprocessing to reduce speech motion artifacts, incorporate a nonspeech communication or language control task into the research design, and conduct a connectivity analysis to adequately assess the impact of functional speech on the hemodynamic response across the perisylvian region.

The Effect of Motion Artifacts on Near-Infrared Spectroscopy (NIRS) Data and Proposal of a Video-NIRS System

AIMS

The aims of this study were (1) to investigate the influence of physical movement on near-infrared spectroscopy (NIRS) data, (2) to establish a video-NIRS system which simultaneously records NIRS data and the subject's movement, and (3) to measure the oxygenated hemoglobin (oxy-Hb) concentration change (Δoxy-Hb) during a word fluency (WF) task.

EXPERIMENT 1

In 5 healthy volunteers, we measured the oxy-Hb and deoxygenated hemoglobin (deoxy-Hb) concentrations during 11 kinds of facial, head, and extremity movements. The probes were set in the bilateral frontal regions. The deoxy-Hb concentration was increased in 85% of the measurements.

EXPERIMENT 2

Using a pillow on the backrest of the chair, we established the video-NIRS system with data acquisition and video capture software. One hundred and seventy-six elderly people performed the WF task. The deoxy-Hb concentration was decreased in 167 subjects (95%).

EXPERIMENT 3

Using the video-NIRS system, we measured the Δoxy-Hb, and compared it with the results of the WF task. Δoxy-Hb was significantly correlated with the number of words.

CONCLUSION

Like the blood oxygen level-dependent imaging effect in functional MRI, the deoxy-Hb concentration will decrease if the data correctly reflect the change in neural activity. The video-NIRS system might be useful to collect NIRS data by recording the waveforms and the subject's appearance simultaneously.

Functional Magnetic Resonance Imaging and Functional Near-Infrared Spectroscopy: Insights from Combined Recording Studies

Although blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) is a widely available, non-invasive technique that offers excellent spatial resolution, it remains limited by practical constraints imposed by the scanner environment. More recently, functional near infrared spectroscopy (fNIRS) has emerged as an alternative hemodynamic-based approach that possesses a number of strengths where fMRI is limited, most notably in portability and higher tolerance for motion. To date, fNIRS has shown promise in its ability to shed light on the functioning of the human brain in populations and contexts previously inaccessible to fMRI. Notable contributions include infant neuroimaging studies and studies examining full-body behaviors, such as exercise. However, much like fMRI, fNIRS has technical constraints that have limited its application to clinical settings, including a lower spatial resolution and limited depth of recording. Thus, by combining fMRI and fNIRS in such a way that the two methods complement each other, a multimodal imaging approach may allow for more complex research paradigms than is feasible with either technique alone. In light of these issues, the purpose of the current review is to: (1) provide an overview of fMRI and fNIRS and their associated strengths and limitations; (2) review existing combined fMRI-fNIRS recording studies; and (3) discuss how their combined use in future research practices may aid in advancing modern investigations of human brain function.

A Gaussian mixture model based adaptive classifier for fNIRS brain-computer interfaces and its testing via simulation

OBJECTIVE

Functional near infra-red spectroscopy (fNIRS) is a promising brain imaging technology for brain-computer interfaces (BCI). Future clinical uses of fNIRS will likely require operation over long time spans, during which neural activation patterns may change. However, current decoders for fNIRS signals are not designed to handle changing activation patterns. The objective of this study is to test via simulations a new adaptive decoder for fNIRS signals, the Gaussian mixture model adaptive classifier (GMMAC).

APPROACH

GMMAC can simultaneously classify and track activation pattern changes without the need for ground-truth labels. This adaptive classifier uses computationally efficient variational Bayesian inference to label new data points and update mixture model parameters, using the previous model parameters as priors. We test GMMAC in simulations in which neural activation patterns change over time and compare to static decoders and unsupervised adaptive linear discriminant analysis classifiers.

MAIN RESULTS

Our simulation experiments show GMMAC can accurately decode under time-varying activation patterns: shifts of activation region, expansions of activation region, and combined contractions and shifts of activation region. Furthermore, the experiments show the proposed method can track the changing shape of the activation region. Compared to prior work, GMMAC performed significantly better than the other unsupervised adaptive classifiers on a difficult activation pattern change simulation: 99% versus  <54% in two-choice classification accuracy.

SIGNIFICANCE

We believe GMMAC will be useful for clinical fNIRS-based brain-computer interfaces, including neurofeedback training systems, where operation over long time spans is required.

Imaging Brain Function with Functional Near-Infrared Spectroscopy in Unconstrained Environments

Assessing the neural correlates of motor and cognitive processes under naturalistic experimentation is challenging due to the movement constraints of traditional brain imaging technologies. The recent advent of portable technologies that are less sensitive to motion artifacts such as Functional Near Infrared Spectroscopy (fNIRS) have been made possible the study of brain function in freely-moving participants. In this paper, we describe a series of proof-of-concept experiments examining the potential of fNIRS in assessing the neural correlates of cognitive and motor processes in unconstrained environments. We show illustrative applications for practicing a sport (i.e., table tennis), playing a musical instrument (i.e., piano and violin) alone or in duo and performing daily activities for many hours (i.e., continuous monitoring). Our results expand upon previous research on the feasibility and robustness of fNIRS to monitor brain hemodynamic changes in different real life settings. We believe that these preliminary results showing the flexibility and robustness of fNIRS measurements may contribute by inspiring future work in the field of applied neuroscience.

The Spatial Release of Cognitive Load in Cocktail Party Is Determined by the Relative Levels of the Talkers

In a multi-talker situation, spatial separation between talkers reduces cognitive processing load: this is the "spatial release of cognitive load". The present study investigated the role played by the relative levels of the talkers on this spatial release of cognitive load. During the experiment, participants had to report the speech emitted by a target talker in the presence of a concurrent masker talker. The spatial separation (0° and 120° angular distance in azimuth) and the relative levels of the talkers (adverse, intermediate, and favorable target-to-masker ratio) were manipulated. The cognitive load was assessed with a prefrontal functional near-infrared spectroscopy. Data from 14 young normal-hearing listeners revealed that the target-to-masker ratio had a direct impact on the spatial release of cognitive load. Spatial separation significantly reduced the prefrontal activity only for the intermediate target-to-masker ratio and had no effect on prefrontal activity for the favorable and the adverse target-to-masker ratios. Therefore, the relative levels of the talkers might be a key point to determine the spatial release of cognitive load and more specifically the prefrontal activity induced by spatial cues in multi-talker situations.

Left Lateral Prefrontal Activity Reflects a Change of Behavioral Tactics to Cope with a Given Rule: An fNIRS Study

Rules prescribe human behavior and our attempts to choose appropriate behavior under a given rule. Cognitive control, a mechanism to choose and evaluate actions under a rule, is required to determine the appropriate behavior within the limitations of that rule. Consequently, such cognitive control increases mental workload. However, the workload caused by a cognitive task might be different when an additional rule must be considered in choosing the action. The present study was a functional near-infrared spectroscopy (fNIRS) investigation of an experimental task, in which the difficulty of an operation and existence of an additional rule were manipulated to dissociate the influence of that additional rule on cognitive processing. Twenty healthy Japanese volunteers participated. The participants performed an experimental task, in which the player caught one of five colored balls from the upper part of a computer screen by operating a mouse. Four task conditions were prepared to manipulate the task difficulty, which was defined in terms of operational difficulty. In turn, operational difficulty was determined by the width of the playable space and the existence of an additional rule, which reduced the score when a red ball was not caught. The 52-channel fNIRS data were collected from the forehead. Two regions of interest (ROIs) associated with the bilateral lateral prefrontal cortices (LPFCs) were determined, and a three-way repeated-measures analysis of variance (ANOVA) was performed using the task-related signal changes from each ROI. The fNIRS results revealed that bilateral LPFCs showed large signal changes with the increase in mental workload. The ANOVA showed a significant interaction between the existence of an additional rule and the location of the ROIs; that is, the left lateral prefrontal area showed a significant increase in signal intensity when the additional rule existed, and the participant occasionally decided to avoid catching a ball to successfully catch the red-colored ball. Thus, activation of the left LPFC corresponded more closely to the increase in cognitive control underlying the behavioral change made to cope with the additional rule.

Functional Near-Infrared Spectroscopy (fNIRS) for Assessing Cerebral Cortex Function During Human Behavior in Natural/Social Situations: A Concise Review

Upon adequate stimulation, real-time maps of cortical hemodynamic responses can be obtained by functional near-infrared spectroscopy (fNIRS), which noninvasively measures changes in oxygenated and deoxygenated hemoglobin after positioning multiple sources and detectors over the human scalp. This review is aimed at giving a concise and simple overview of the basic principles of fNIRS including features, strengths, advantages, limitations, and utility for evaluating human behavior. The transportable/wireless commercially available fNIRS systems have a time resolution of 1 to 10 Hz, a depth sensitivity of about 1.5 cm, and a spatial resolution up to 1 cm. fNIRS has been found suitable for many applications on human beings, either adults or infants/children, in the field of social sciences, neuroimaging basic research, and medicine. Some examples of present and future prospects of fNIRS for assessing cerebral cortex function during human behavior in different situations (in natural and social situations) will be provided. Moreover, the most recent fNIRS studies for investigating interpersonal interactions by adopting the hyperscanning approach, which consists of the measurement of brain activity simultaneously on two or more people, will be reported.

Using affective brain-computer interfaces to characterize human influential factors for speech quality-of-experience perception modelling

As new speech technologies emerge, telecommunication service providers have to provide superior user experience in order to remain competitive. To this end, quality-of-experience (QoE) perception modelling and measurement has become a key priority. QoE models rely on three influence factors: technological, contextual and human. Existing solutions have typically relied on the former two and human influence factors (HIFs) have been mostly neglected due to difficulty in measuring them. In this paper, we show that measuring human affective states is important for QoE measurement and propose the use of affective brain-computer interfaces (aBCIs) for objective measurement of perceived QoE for two emerging speech technologies, namely far-field hands-free communications and text-to-speech systems. When incorporating subjectively-derived HIFs into the QoE model, gains of up to 26.3 % could be found relative to utilizing only technological factors. When utilizing HIFs derived from an electroencephalography (EEG) based aBCI, in turn, gains of up to 14.5 % were observed. These findings show the importance of using aBCIs in QoE measurement and also highlight that further improvement may be warranted once improved affective state correlates are found from EEGs and/or other neurophysiological modalities.

Neurovascular coupling in humans: Physiology, methodological advances and clinical implications

Neurovascular coupling reflects the close temporal and regional linkage between neural activity and cerebral blood flow. Although providing mechanistic insight, our understanding of neurovascular coupling is largely limited to non-physiologicalex vivopreparations and non-human models using sedatives/anesthetics with confounding cerebrovascular implications. Herein, with particular focus on humans, we review the present mechanistic understanding of neurovascular coupling and highlight current approaches to assess these responses and the application in health and disease. Moreover, we present new guidelines for standardizing the assessment of neurovascular coupling in humans. To improve the reliability of measurement and related interpretation, the utility of new automated software for neurovascular coupling is demonstrated, which provides the capacity for coalescing repetitive trials and time intervals into single contours and extracting numerous metrics (e.g., conductance and pulsatility, critical closing pressure, etc.) according to patterns of interest (e.g., peak/minimum response, time of response, etc.). This versatile software also permits the normalization of neurovascular coupling metrics to dynamic changes in arterial blood gases, potentially influencing the hyperemic response. It is hoped that these guidelines, combined with the newly developed and openly available software, will help to propel the understanding of neurovascular coupling in humans and also lead to improved clinical management of this critical physiological function.

Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Functional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.

Processing Functional Near Infrared Spectroscopy Signal with a Kalman Filter to Assess Working Memory during Simulated Flight

Working memory (WM) is a key executive function for operating aircraft, especially when pilots have to recall series of air traffic control instructions. There is a need to implement tools to monitor WM as its limitation may jeopardize flight safety. An innovative way to address this issue is to adopt a Neuroergonomics approach that merges knowledge and methods from Human Factors, System Engineering, and Neuroscience. A challenge of great importance for Neuroergonomics is to implement efficient brain imaging techniques to measure the brain at work and to design Brain Computer Interfaces (BCI). We used functional near infrared spectroscopy as it has been already successfully tested to measure WM capacity in complex environment with air traffic controllers (ATC), pilots, or unmanned vehicle operators. However, the extraction of relevant features from the raw signal in ecological environment is still a critical issue due to the complexity of implementing real-time signal processing techniques without a priori knowledge. We proposed to implement the Kalman filtering approach, a signal processing technique that is efficient when the dynamics of the signal can be modeled. We based our approach on the Boynton model of hemodynamic response. We conducted a first experiment with nine participants involving a basic WM task to estimate the noise covariances of the Kalman filter. We then conducted a more ecological experiment in our flight simulator with 18 pilots who interacted with ATC instructions (two levels of difficulty). The data was processed with the same Kalman filter settings implemented in the first experiment. This filter was benchmarked with a classical pass-band IIR filter and a Moving Average Convergence Divergence (MACD) filter. Statistical analysis revealed that the Kalman filter was the most efficient to separate the two levels of load, by increasing the observed effect size in prefrontal areas involved in WM. In addition, the use of a Kalman filter increased the performance of the classification of WM levels based on brain signal. The results suggest that Kalman filter is a suitable approach for real-time improvement of near infrared spectroscopy signal in ecological situations and the development of BCI.

Passive BCI based on drowsiness detection: an fNIRS study

We use functional near-infrared spectroscopy (fNIRS) to discriminate the alert and drowsy states for a passive brain-computer interface (BCI). The passive brain signals for the drowsy state are acquired from the prefrontal and dorsolateral prefrontal cortex. The experiment is performed on 13 healthy subjects using a driving simulator, and their brain activity is recorded using a continuous-wave fNIRS system. Linear discriminant analysis (LDA) is employed for training and testing, using the data from the prefrontal, left- and right-dorsolateral prefrontal regions. For classification, eight features are tested: mean oxyhemoglobin, mean deoxyhemoglobin, skewness, kurtosis, signal slope, number of peaks, sum of peaks, and signal peak, in 0~5, 0~10, and 0~15 second time windows, respectively. The results show that the best performance for classification is achieved using mean oxyhemoglobin, the signal peak, and the sum of peaks as features. The average accuracies in the right dorsolateral prefrontal cortex (83.1, 83.4 and 84.9% in the 0~5, 0~10 and 0~15 second time windows, respectively) show that the proposed method has an effective utility for detection of drowsiness for a passive BCI.

Correlates of Near-Infrared Spectroscopy Brain-Computer Interface Accuracy in a Multi-Class Personalization Framework

Brain–computer interfaces (BCIs) provide individuals with a means of interacting with a computer using only neural activity. To date, the majority of near-infrared spectroscopy (NIRS) BCIs have used prescribed tasks to achieve binary control. The goals of this study were to evaluate the possibility of using a personalized approach to establish control of a two-, three-, four-, and five-class NIRS–BCI, and to explore how various user characteristics correlate to accuracy. Ten able-bodied participants were recruited for five data collection sessions. Participants performed six mental tasks and a personalized approach was used to select each individual’s best discriminating subset of tasks. The average offline cross-validation accuracies achieved were 78, 61, 47, and 37% for the two-, three-, four-, and five-class problems, respectively. Most notably, all participants exceeded an accuracy of 70% for the two-class problem, and two participants exceeded an accuracy of 70% for the three-class problem. Additionally, accuracy was found to be strongly positively correlated (Pearson’s) with perceived ease of session (ρ = 0.653), ease of concentration (ρ = 0.634), and enjoyment (ρ = 0.550), but strongly negatively correlated with verbal IQ (ρ = −0.749).

Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls

Estimating cognitive or affective state from neurophysiological signals and designing applications that make use of this information requires expertise in many disciplines such as neurophysiology, machine learning, experimental psychology, and human factors. This makes it difficult to perform research that is strong in all its aspects as well as to judge a study or application on its merits. On the occasion of the special topic "Using neurophysiological signals that reflect cognitive or affective state" we here summarize often occurring pitfalls and recommendations on how to avoid them, both for authors (researchers) and readers. They relate to defining the state of interest, the neurophysiological processes that are expected to be involved in the state of interest, confounding factors, inadvertently "cheating" with classification analyses, insight on what underlies successful state estimation, and finally, the added value of neurophysiological measures in the context of an application. We hope that this paper will support the community in producing high quality studies and well-validated, useful applications.

Real-time state estimation in a flight simulator using fNIRS

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot's instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot's mental state matched significantly better than chance with the pilot's real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development.

Characterization of mind wandering using fNIRS

Assessing whether someone is attending to a task has become important for educational and professional applications. Such attentional drifts are usually termed mind wandering (MW). The purpose of the current study is to test to what extent a recent neural imaging modality can be used to detect MW episodes. Functional near infrared spectroscopy is a non-invasive neuroimaging technique that has never been used so far to measure MW. We used the Sustained Attention to Response Task (SART) to assess when subjects attention leaves a primary task. Sixteen-channel fNIRS data were collected over frontal cortices. We observed significant activations over the medial prefrontal cortex (mPFC) during MW, a brain region associated with the default mode network (DMN). fNIRS data were used to classify MW data above chance level. In line with previous brain-imaging studies, our results confirm the ability of fNIRS to detect Default Network activations in the context of MW.

Cortical cross-modal plasticity following deafness measured using functional near-infrared spectroscopy

Evidence from functional neuroimaging studies suggests that the auditory cortex can become more responsive to visual and somatosensory stimulation following deafness, and that this occurs predominately in the right hemisphere. Extensive cross-modal plasticity in prospective cochlear implant recipients is correlated with poor speech outcomes following implantation, highlighting the potential impact of central auditory plasticity on subsequent aural rehabilitation. Conversely, the effects of hearing restoration with a cochlear implant on cortical plasticity are less well understood, since the use of most neuroimaging techniques in CI recipients is either unsafe or problematic due to the electromagnetic artefacts generated by CI stimulation. Additionally, techniques such as functional magnetic resonance imaging (fMRI) are confounded by acoustic noise produced by the scanner that will be perceived more by hearing than by deaf individuals. Subsequently it is conceivable that auditory responses to acoustic noise produced by the MR scanner may mask auditory cortical responses to non-auditory stimulation, and render inter-group comparisons less significant. Uniquely, functional near-infrared spectroscopy (fNIRS) is a silent neuroimaging technique that is non-invasive and completely unaffected by the presence of a CI. Here, we used fNIRS to study temporal-lobe responses to auditory, visual and somatosensory stimuli in thirty profoundly-deaf participants and thirty normally-hearing controls. Compared with silence, acoustic noise stimuli elicited a significant group fNIRS response in the temporal region of normally-hearing individuals, which was not seen in profoundly-deaf participants. Visual motion elicited a larger group response within the right temporal lobe of profoundly-deaf participants, compared with normally-hearing controls. However, bilateral temporal lobe fNIRS activation to somatosensory stimulation was comparable in both groups. Using fNIRS these results confirm that auditory deprivation is associated with cross-modal plasticity of visual inputs to auditory cortex. Although we found no evidence for plasticity of somatosensory inputs, it is possible that our recordings may have included activation of somatosensory cortex that masked any group differences in auditory cortical responses due to the limited spatial resolution associated with fNIRS.

Challenges in clinical applications of brain computer interfaces in individuals with spinal cord injury

Brain computer interfaces (BCIs) are devices that measure brain activities and translate them into control signals used for a variety of applications. Among them are systems for communication, environmental control, neuroprostheses, exoskeletons, or restorative therapies. Over the last years the technology of BCIs has reached a level of matureness allowing them to be used not only in research experiments supervised by scientists, but also in clinical routine with patients with neurological impairments supervised by clinical personnel or caregivers. However, clinicians and patients face many challenges in the application of BCIs. This particularly applies to high spinal cord injured patients, in whom artificial ventilation, autonomic dysfunctions, neuropathic pain, or the inability to achieve a sufficient level of control during a short-term training may limit the successful use of a BCI. Additionally, spasmolytic medication and the acute stress reaction with associated episodes of depression may have a negative influence on the modulation of brain waves and therefore the ability to concentrate over an extended period of time. Although BCIs seem to be a promising assistive technology for individuals with high spinal cord injury systematic investigations are highly needed to obtain realistic estimates of the percentage of users that for any reason may not be able to operate a BCI in a clinical setting.