fNIRS-based brain-computer interfaces: a review

Activation of Prefrontal Cortex in Process of Oral and Finger Shape Discrimination: fNIRS Study

Background

The differences in the brain activities of the insular and the visual association cortices have been reported between oral and manual stereognosis. However, these results were not conclusive because of the inherent differences in the task performance-related motor sequence conditions. We hypothesized that the involvement of the prefrontal cortex may be different between finger and oral shape discrimination. This study was conducted to clarify temporal changes in prefrontal activities occurring in the processes of oral and finger tactual shape discrimination using prefrontal functional near-infrared spectroscopy (fNIRS).

Methods

Six healthy right-handed males [aged 30.8 ± 8.2 years (mean ± SD)] were enrolled. Measurements of prefrontal activities were performed using a 22-channel fNIRS device (ETG-100, Hitachi Medical Co., Chiba, Japan) during experimental blocks that included resting state (REST), nonsense shape discrimination (SHAM), and shape discrimination (SHAPE).

Results

No significant difference was presented with regard to the number of correct answers during trials between oral and finger SHAPE discrimination. Additionally, a statistical difference for the prefrontal fNIRS activity between oral and finger shape discrimination was noted in CH 1. Finger SHAPE, as compared with SHAM, presented a temporally shifting onset and burst in the prefrontal activities from the frontopolar area (FPA) to the orbitofrontal cortex (OFC). In contrast, oral SHAPE as compared with SHAM was shown to be temporally overlapped in the onset and burst of the prefrontal activities in the dorsolateral prefrontal cortex (DLPFC)/FPA/OFC.

Conclusion

The prefrontal activities temporally shifting from the FPA to the OFC during SHAPE as compared with SHAM may suggest the segregated serial prefrontal processing from the manipulation of a target image to the decision making during the process of finger shape discrimination. In contrast, the temporally overlapped prefrontal activities of the DLPFC/FPA/OFC in the oral SHAPE block may suggest the parallel procession of the repetitive involvement of generation, manipulation, and decision making in order to form a reliable representation of target objects.

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Using Transcranial Near-Infrared Photobiomodulation With the Same Device

Transcranial near-infrared photobiomodulation (tNIR-PBM) can modulate physiological characteristics of the human brain, such as the cerebral blood flow and oxidative metabolism. Here, we investigated whether the performance of near-infrared spectroscopy (NIRS)-based brain-computer interfaces (BCIs) can be improved by tNIR-PBM applied to the prefrontal cortex with the same NIRS device. A total of 14 healthy individuals participated in the NIRS-based BCI study where the aim was to distinguish the mental arithmetic task from the idle state (IS) task either after tNIR-PBM or after sham stimulation, with the two experiments being conducted at least two days apart. The tNIR-PBM was applied by simply turning on the NIRS recording equipment for 20 min. To evaluate the degree of performance improvement obtained after tNIR-PBM, the average BCI classification accuracy obtained under the tNIR-PBM condition was compared with that obtained under the sham stimulation condition. The classification accuracy of NIRS-based BCI was significantly improved upon conduction of tNIR-PBM (82.74%) as compared to that in the sham stimulation condition (76.07%, p < 0.005). Thus, our results suggest that simply turning on the NIRS recording equipment before the BCI experiment can improve the performance of the NIRS-based BCI system.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain-computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

Decoding Three Different Preference Levels of Consumers Using Convolutional Neural Network: A Functional Near-Infrared Spectroscopy Study

This study decodes consumers' preference levels using a convolutional neural network (CNN) in neuromarketing. The classification accuracy in neuromarketing is a critical factor in evaluating the intentions of the consumers. Functional near-infrared spectroscopy (fNIRS) is utilized as a neuroimaging modality to measure the cerebral hemodynamic responses. In this study, a specific decoding structure, called CNN-based fNIRS-data analysis, was designed to achieve a high classification accuracy. Compared to other methods, the automated characteristics, constant training of the dataset, and learning efficiency of the proposed method are the main advantages. The experimental procedure required eight healthy participants (four female and four male) to view commercial advertisement videos of different durations (15, 30, and 60 s). The cerebral hemodynamic responses of the participants were measured. To compare the preference classification performances, CNN was utilized to extract the most common features, including the mean, peak, variance, kurtosis, and skewness. Considering three video durations, the average classification accuracies of 15, 30, and 60 s videos were 84.3, 87.9, and 86.4%, respectively. Among them, the classification accuracy of 87.9% for 30 s videos was the highest. The average classification accuracies of three preferences in females and males were 86.2 and 86.3%, respectively, showing no difference in each group. By comparing the classification performances in three different combinations (like vs. so-so, like vs. dislike, and so-so vs. dislike) between two groups, male participants were observed to have targeted preferences for commercial advertising, and the classification performance 88.4% between "like" vs. "dislike" out of three categories was the highest. Finally, pairwise classification performance are shown as follows: For female, 86.1% (like vs. so-so), 87.4% (like vs. dislike), 85.2% (so-so vs. dislike), and for male 85.7, 88.4, 85.1%, respectively.

Neural Correlates of Mental Rotation in Preschoolers With High or Low Working Memory Capacity: An fNIRS Study

This study explored the differentiated neural correlates of mental rotation (MR) in preschoolers with high and low working memory capacity using functional near-infrared spectroscopy (fNIRS). Altogether 38 Chinese preschoolers (M = 5.0 years, SD = 0.69 years) completed the Working Memory Capacity (WMC) test, the Mental Rotation (MR), and its Control tasks (without MR). They were divided into High-WMC (N₁ = 9) and Low-WMC (N₂ = 18) groups based on the WMC scores. The behavioral and fNIRS results indicated that: (1) there were no significant differences in MR task performance between the High-WMC (M_mr = 23.44, SD = 0.88) and Low-WMC group (M_mr = 23.67, SD = 0.59); (2) the Low-WMC group activated BA6, BA8, BA 9, and BA 44, whereas the High-WMC group activated BA8, BA10 and BA 44 during mental rotation; (3) significant differences were found in the activation of BA44 and BA9 between the High-WMC and Low-WMC groups during mental rotation; and (4) the High-WMC and Low-WMC groups differed significantly in the activation of BA 9 and BA10 during the control tasks, indicating that both areas might be responsible for the group differences in working memory.

Prefrontal Asymmetry BCI Neurofeedback Datasets

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Hybrid EEG-fNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD)

Brain-computer interface (BCI) multi-modal fusion has the potential to generate multiple commands in a highly reliable manner by alleviating the drawbacks associated with single modality. In the present work, a hybrid EEG-fNIRS BCI system—achieved through a fusion of concurrently recorded electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals—is used to overcome the limitations of uni-modality and to achieve higher tasks classification. Although the hybrid approach enhances the performance of the system, the improvements are still modest due to the lack of availability of computational approaches to fuse the two modalities. To overcome this, a novel approach is proposed using Multi-resolution singular value decomposition (MSVD) to achieve system- and feature-based fusion. The two approaches based up different features set are compared using the KNN and Tree classifiers. The results obtained through multiple datasets show that the proposed approach can effectively fuse both modalities with improvement in the classification accuracy.

Enhancing Classification Performance of fNIRS-BCI by Identifying Cortically Active Channels Using the z-Score Method

A state-of-the-art brain–computer interface (BCI) system includes brain signal acquisition, noise removal, channel selection, feature extraction, classification, and an application interface. In functional near-infrared spectroscopy-based BCI (fNIRS-BCI) channel selection may enhance classification performance by identifying suitable brain regions that contain brain activity. In this study, the z-score method for channel selection is proposed to improve fNIRS-BCI performance. The proposed method uses cross-correlation to match the similarity between desired and recorded brain activity signals, followed by forming a vector of each channel’s correlation coefficients’ maximum values. After that, the z-score is calculated for each value of that vector. A channel is selected based on a positive z-score value. The proposed method is applied to an open-access dataset containing mental arithmetic (MA) and motor imagery (MI) tasks for twenty-nine subjects. The proposed method is compared with the conventional t-value method and with no channel selected, i.e., using all channels. The z-score method yielded significantly improved (p < 0.0167) classification accuracies of 87.2 ± 7.0%, 88.4 ± 6.2%, and 88.1 ± 6.9% for left motor imagery (LMI) vs. rest, right motor imagery (RMI) vs. rest, and mental arithmetic (MA) vs. rest, respectively. The proposed method is also validated on an open-access database of 17 subjects, containing right-hand finger tapping (RFT), left-hand finger tapping (LFT), and dominant side foot tapping (FT) tasks.The study shows an enhanced performance of the z-score method over the t-value method as an advancement in efforts to improve state-of-the-art fNIRS-BCI systems’ performance.

Amplitude of fNIRS Resting-State Global Signal Is Related to EEG Vigilance Measures: A Simultaneous fNIRS and EEG Study

Recently, functional near-infrared spectroscopy (fNIRS) has been utilized to image the hemodynamic activities and connectivity in the human brain. With the advantage of economic efficiency, portability, and fewer physical constraints, fNIRS enables studying of the human brain at versatile environment and various body positions, including at bed side and during exercise, which complements the use of functional magnetic resonance imaging (fMRI). However, like fMRI, fNIRS imaging can be influenced by the presence of a strong global component. Yet, the nature of the global signal in fNIRS has not been established. In this study, we investigated the relationship between fNIRS global signal and electroencephalogram (EEG) vigilance using simultaneous recordings in resting healthy subjects in high-density and whole-head montage. In Experiment 1, data were acquired at supine, sitting, and standing positions. Results found that the factor of body positions significantly affected the amplitude of the resting-state fNIRS global signal, prominently in the frequency range of 0.05-0.1 Hz but not in the very low frequency range of less than 0.05 Hz. As a control, the task-induced fNIRS or EEG responses to auditory stimuli did not differ across body positions. However, EEG vigilance plays a modulatory role in the fNIRS signals in the frequency range of less than 0.05 Hz: resting-state sessions of low EEG vigilance measures are associated with high amplitudes of fNIRS global signals. Moreover, in Experiment 2, we further examined the epoch-to-epoch fluctuations in concurrent fNIRS and EEG data acquired from a separate group of subjects and found a negative temporal correlation between EEG vigilance measures and fNIRS global signal amplitudes. Our study for the first time revealed that vigilance as a neurophysiological factor modulates the resting-state dynamics of fNIRS, which have important implications for understanding and processing the noises in fNIRS signals.

Measuring dlPFC Signals to Predict the Success of Merchandising Elements at the Point-of-Sale - A fNIRS Approach

The (re-)launch of products is frequently accompanied by point-of-sale (PoS) marketing campaigns in order to foster sales. Predicting the success of these merchandising elements at the PoS on sales is of interest to research and practice, as the misinvestments that are based on the fragmented PoS literature are tremendous. Likewise, the predictive power of neuropsychological methods has been demonstrated in various research work. Nevertheless, the practical application of these neuropsychological methods is still limited. In order to foster the application of neuropsychological methods in research and practice, the current research work aims to explore, whether mobile functional near-infrared spectroscopy (fNIRS) - as a portable neuroimaging method - has the potential to predict the success of PoS merchandising elements by rendering significant neural signatures of brain regions of the dorsolateral prefrontal cortex (dlPFC), highlighting its potential to forecast shoppers' behaviour aka sales at the PoS. Building on previous research findings, the results of the given research work indicate that the neural signal of brain regions of the dlPFC, measured with mobile fNIRS, is able to predict actual sales associated with PoS merchandising elements, relying on the cortical relief effect. More precisely, the research findings support the hypothesis that the reduced neural activity of brain regions associated with the dlPFC can predict sales at the PoS, emphasising another crucial neural signature to predict shoppers' purchase behaviour, next to the frequently cited reward association system. The research findings offer an innovative perspective on how to design and evaluate PoS merchandising elements, indicating fruitful theoretical and practical implications.

A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers

Brain signals refer to the biometric information collected from the human brain. The research on brain signals aims to discover the underlying neurological or physical status of the individuals by signal decoding. The emerging deep learning techniques have improved the study of brain signals significantly in recent years. In this work, we first present a taxonomy of non-invasive brain signals and the basics of deep learning algorithms. Then, we provide a comprehensive survey of the frontiers of applying deep learning for non-invasive brain signals analysis, by summarizing a large number of recent publications. Moreover, upon the deep learning-powered brain signal studies, we report the potential real-world applications which benefit not only disabled people but also normal individuals. Finally, we discuss the opening challenges and future directions.

Performance Improvement for Detecting Brain Function Using fNIRS: A Multi-Distance Probe Configuration With PPL Method

To improve the spatial resolution of imaging and get more effective brain function information, a multi-distance probe configuration with three distances (28.2, 40, and 44.7 mm) and 52 channels is designed. At the same time, a data conversion method of modified Beer–Lambert law (MBLL) with partial pathlength (PPL) is proposed. In the experiment, three kinds of tasks, grip of left hand, grip of right hand, and rest, are performed with eight healthy subjects. First, with a typical single-distance probe configuration (30 mm, 24 channels), the feasibility of the proposed MBLL with PPL is preliminarily validated. Further, the characteristic of the proposed method is evaluated with the multi-distance probe configuration. Compared with MBLL with differential pathlength factor (DPF), the proposed MBLL with PPL is able to acquire more obvious concentration change and can achieve higher classification accuracy of the three tasks. Then, with the proposed method, the performance of the multi-distance probe configuration is discussed. Results show that, compared with a single distance, the combination of the three distances has better spatial resolution and could explore more accurate brain activation information. Besides, the classification accuracy of the three tasks obtained with the combination of three distances is higher than that of any combination of two distances. Also, with the combination of the three distances, the two-class classification between different tasks is carried out. Both theory and experimental results demonstrate that, using multi-distance probe configuration and the MBLL with PPL method, the performance of brain function detected by NIRS can be improved.

Wearable hip-assist robot modulates cortical activation during gait in stroke patients: a functional near-infrared spectroscopy study

Background

Gait dysfunction is common in post-stroke patients as a result of impairment in cerebral gait mechanism. Powered robotic exoskeletons are promising tools to maximize neural recovery by delivering repetitive walking practice.

Objectives

The purpose of this study was to investigate the modulating effect of the Gait Enhancing and Motivating System-Hip (GEMS-H) on cortical activation during gait in patients with chronic stroke. Methods. Twenty chronic stroke patients performed treadmill walking at a self-selected speed either with assistance of GEMS-H (GEMS-H) or without assistance of GEMS-H (NoGEMS-H). Changes in oxygenated hemoglobin (oxyHb) concentration in the bilateral primary sensorimotor cortex (SMC), premotor cortices (PMC), supplemental motor areas (SMA), and prefrontal cortices (PFC) were recorded using functional near infrared spectroscopy.

Results

Walking with the GEMS-H promoted symmetrical SMC activation, with more activation in the affected hemisphere than in NoGEMS-H conditions. GEMS-H also decreased oxyHb concentration in the late phase over the ipsilesional SMC and bilateral SMA (P < 0.05).

Conclusions

The results of the present study reveal that the GEMS-H promoted more SMC activation and a balanced activation pattern that helped to restore gait function. Less activation in the late phase over SMC and SMA during gait with GEMS-H indicates that GEMS-H reduces the cortical participation of stroke gait by producing rhythmic hip flexion and extension movement and allows a more coordinate and efficient gait patterns.

Trial registration NCT03048968. Registered 06 Feb 2017

Therapeutic benefits of music-based synchronous finger tapping in Parkinson's disease-an fNIRS study protocol for randomized controlled trial in Dalian, China

BACKGROUND

Music therapy improves neuronal activity and connectivity of healthy persons and patients with clinical symptoms of neurological diseases like Parkinson's disease, Alzheimer's disease, and major depression. Despite the plethora of publications that have reported the positive effects of music interventions, little is known about how music improves neuronal activity and connectivity in afflicted patients.

METHODS

For patients suffering from Parkinson's disease (PD), we propose a daily 25-min music-based synchronous finger tapping (SFT) intervention for 8 weeks. Eligible participants with PD are split into two groups: an intervention group and a control arm. In addition, a third cohort of healthy controls will be recruited. Assessment of finger tapping performances, the Unified Parkinson's Disease Rating Scale (UPDRS), an n-back test, the Montreal Cognitive Assessment (MoCA), as well as oxygenated hemoglobin (HbO2), deoxygenated hemoglobin (HbR), and total hemoglobin activation collected by functional near-infrared spectroscopy (fNIRS) are measured at baseline, week 4 (during), week 8 (post), and week 12 (retention) of the study. Data collected from the two PD groups are compared to baseline performances from healthy controls.

DISCUSSION

This exploratory prospective trial study investigates the cortical neuronal activity and therapeutic effects associated with an auditory external cue used to induce automatic and implicit synchronous finger tapping in patients diagnosed with PD. The extent to which the intervention is effective may be dependent on the severity of the disease. The study's findings are used to inform larger clinical studies for optimization and further exploration of the therapeutic effects of movement-based music therapy on neural activity in neurological diseases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04212897 . Registered on December 30, 2019. The participant recruitment and study protocol have received ethical approval from the First Affiliated Hospital of Dalian Medical University. The hospital Protocol Record number is PJ-KY-2019-123. The protocol was named "fNIRS Studies of Music Intervention of Parkinson's Disease." The current protocol is version 1.1, revised on September 1, 2020.

Transcranial photobiomodulation-induced changes in human brain functional connectivity and network metrics mapped by whole-head functional near-infrared spectroscopy in vivo

Transcranial photobiomodulation (tPBM) with near-infrared light on the human head has been shown to enhance human cognition. In this study, tPBM-induced effects on resting state brain networks were investigated using 111-channel functional near-infrared spectroscopy over the whole head. Measurements were collected with and without 8-minute tPBM in 19 adults. Functional connectivity (FC) and brain network metrics were quantified using Pearson's correlation coefficients and graph theory analysis (GTA), respectively, for the periods of pre-, during, and post-tPBM. Our results revealed that tPBM (1) enhanced information processing speed and efficiency of the brain network, and (2) increased FC significantly in the frontal-parietal network, shedding light on a better understanding of tPBM effects on brain networks.

A Unified Analytical Framework With Multiple fNIRS Features for Mental Workload Assessment in the Prefrontal Cortex

Knowing the actual level of mental workload is important to ensure the efficacy of brain-computer interface (BCI) based cognitive training. Extracting signals from limited area of a brain region might not reveal the actual information. In this study, a functional near-infrared spectroscopy (fNIRS) device equipped with multi-channel and multi-distance measurement capability was employed for the development of an analytical framework to assess mental workload in the prefrontal cortex (PFC). In addition to the conventional features, e.g. hemodynamic slope, we introduced a new feature - deep contribution ratio which is the proportion of cerebral hemodynamics to the fNIRS signals. Multiple sets of features were examined by a simple logical operator to suppress the false detection rate in identifying the activated channels. Using the number of activated channels as input to a linear support vector machine (SVM), the performance of the proposed analytical framework was assessed in classifying three levels of mental workload. The best set of features involves the combination of hemodynamic slope and deep contribution ratio, where the identified number of activated channels returned an average accuracy of 80.6% in predicting mental workload, compared to a single conventional feature (accuracy: 59.8%). This suggests the feasibility of the proposed analytical framework with multiple features as a means towards a more accurate assessment of mental workload in fNIRS-based BCI applications.

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

The goal of this study was to develop and validate a hybrid brain-computer interface (BCI) system for home automation control. Over the past decade, BCIs represent a promising possibility in the field of medical (e.g., neuronal rehabilitation), educational, mind reading, and remote communication. However, BCI is still difficult to use in daily life because of the challenges of the unfriendly head device, lower classification accuracy, high cost, and complex operation. In this study, we propose a hybrid BCI system for home automation control with two brain signals acquiring electrodes and simple tasks, which only requires the subject to focus on the stimulus and eye blink. The stimulus is utilized to select commands by generating steady-state visually evoked potential (SSVEP). The single eye blinks (i.e., confirm the selection) and double eye blinks (i.e., deny and re-selection) are employed to calibrate the SSVEP command. Besides that, the short-time Fourier transform and convolution neural network algorithms are utilized for feature extraction and classification, respectively. The results show that the proposed system could provide 38 control commands with a 2 s time window and a good accuracy (i.e., 96.92%) using one bipolar electroencephalogram (EEG) channel. This work presents a novel BCI approach for the home automation application based on SSVEP and eye blink signals, which could be useful for the disabled. In addition, the provided strategy of this study—a friendly channel configuration (i.e., one bipolar EEG channel), high accuracy, multiple commands, and short response time—might also offer a reference for the other BCI controlled applications.

Recent Developments in Instrumentation of Functional Near-Infrared Spectroscopy Systems

In the last three decades, the development and steady improvement of various optical technologies at the near-infrared region of the electromagnetic spectrum has inspired a large number of scientists around the world to design and develop functional near-infrared spectroscopy (fNIRS) systems for various medical applications. This has been driven further by the availability of new sources and detectors that support very compact and wearable system designs. In this article, we review fNIRS systems from the instrumentation point of view, discussing the associated challenges and state-of-the-art approaches. In the beginning, the fundamentals of fNIRS systems as well as light-tissue interaction at NIR are briefly introduced. After that, we present the basics of NIR systems instrumentation. Next, the recent development of continuous-wave, frequency-domain, and time-domain fNIRS systems are discussed. Finally, we provide a summary of these three modalities and an outlook into the future of fNIRS technology.

Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing

Emotions play a critical role in our daily lives, so the understanding and recognition of emotional responses is crucial for human research. Affective computing research has mostly used non-immersive two-dimensional (2D) images or videos to elicit emotional states. However, immersive virtual reality, which allows researchers to simulate environments in controlled laboratory conditions with high levels of sense of presence and interactivity, is becoming more popular in emotion research. Moreover, its synergy with implicit measurements and machine-learning techniques has the potential to impact transversely in many research areas, opening new opportunities for the scientific community. This paper presents a systematic review of the emotion recognition research undertaken with physiological and behavioural measures using head-mounted displays as elicitation devices. The results highlight the evolution of the field, give a clear perspective using aggregated analysis, reveal the current open issues and provide guidelines for future research.

A Computationally Efficient Method for Hybrid EEG-fNIRS BCI Based on the Pearson Correlation

A hybrid brain computer interface (BCI) system considered here is a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG-fNIRS signals are simultaneously recorded to achieve high motor imagery task classification. This integration helps to achieve better system performance, but at the cost of an increase in system complexity and computational time. In hybrid BCI studies, channel selection is recognized as the key element that directly affects the system's performance. In this paper, we propose a novel channel selection approach using the Pearson product-moment correlation coefficient, where only highly correlated channels are selected from each hemisphere. Then, four different statistical features are extracted, and their different combinations are used for the classification through KNN and Tree classifiers. As far as we know, there is no report available that explored the Pearson product-moment correlation coefficient for hybrid EEG-fNIRS BCI channel selection. The results demonstrate that our hybrid system significantly reduces computational burden while achieving a classification accuracy with high reliability comparable to the existing literature.

Modified Support Vector Machine for Detecting Stress Level Using EEG Signals

Stress is categorized as a condition of mental strain or pressure approaches because of upsetting or requesting conditions. There are various sources of stress initiation. Researchers consider human cerebrum as the primary wellspring of stress. To study how each individual encounters stress in different forms, researchers conduct surveys and monitor it. The paper presents the fusion of 5 algorithms to enhance the accuracy for detection of mental stress using EEG signals. The Whale Optimization Algorithm has been modified to select the optimal kernel in the SVM classifier for stress detection. An integrated set of algorithms (NLM, DCT, and MBPSO) has been used for preprocessing, feature extraction, and selection. The proposed algorithm has been tested on EEG signals collected from 14 subjects to identify the stress level. The proposed approach was validated using accuracy, sensitivity, specificity, and F1 score with values of 96.36%, 96.84%, 90.8%, and 97.96% and was found to be better than the existing ones. The algorithm may be useful to psychiatrists and health consultants for diagnosing the stress level.

Reinforcement-Sensitive Personality Traits Associated With Passion in Heterosexual Intimate Relationships: An fNIRS Investigation

According to the triangular theory of love, passion is an indispensable component of romantic love. Some brain imaging studies have shown that passionate arousal in intimate relationships is associated with the reward circuits in the brain. We hypothesized that the individual reward sensitivity trait is also related to passion in intimate relationships, and two separate studies were conducted in the present research. In the first study, 558 college students who were currently in love were selected as participants. The correlation between intimacy and reinforcement sensitivity in individuals identifying as heterosexual was explored using the Sensitivity to Punishment and Sensitivity to Reward Questionnaire, the Passionate Love Scale, and the Triangular Love Scale. In the second study, participants were 42 college students who were also currently in love. Functional near-infrared spectroscopy (fNIRS) was adopted to explore the neurophysiological interaction between reward sensitivity and emotional arousal induced in participants when presented a photograph of their partner, a friend, or a stranger. The results showed that reward sensitivity was positively correlated with passion, and punishment sensitivity was negatively correlated with intimacy and commitment. Significant interactions between reward sensitivity and photograph type were found, and the triangular part of the inferior frontal gyrus showed a particular relevance to the reward-sensitive personality trait toward partners. Overall, the findings support reinforcement sensitivity theory and suggest that reinforcement-sensitive personality traits (personality traits of reward and punishment sensitivity) are associated with all three components of love, with only reward sensitivity being related to passion.

A Systematic Review of Cerebral Functional Near-Infrared Spectroscopy in Chronic Neurological Diseases-Actual Applications and Future Perspectives

BACKGROUND

The management of people affected by age-related neurological disorders requires the adoption of targeted and cost-effective interventions to cope with chronicity. Therapy adaptation and rehabilitation represent major targets requiring long-term follow-up of neurodegeneration or, conversely, the promotion of neuroplasticity mechanisms. However, affordable and reliable neurophysiological correlates of cerebral activity to be used throughout treatment stages are often lacking. The aim of this systematic review is to highlight actual applications of functional Near-Infrared Spectroscopy (fNIRS) as a versatile optical neuroimaging technology for investigating cortical hemodynamic activity in the most common chronic neurological conditions.

METHODS

We reviewed studies investigating fNIRS applications in Parkinson's Disease (PD), Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) as those focusing on motor and cognitive impairment in ageing and Multiple Sclerosis (MS) as the most common chronic neurological disease in young adults. The literature search was conducted on NCBI PubMed and Web of Science databases by PRISMA guidelines.

RESULTS

We identified a total of 63 peer-reviewed articles. The AD spectrum is the most investigated pathology with 40 articles ranging from the traditional monitoring of tissue oxygenation to the analysis of functional resting-state conditions or cognitive functions by means of memory and verbal fluency tasks. Conversely, applications in PD (12 articles) and MS (11 articles) are mainly focused on the characterization of motor functions and their association with dual-task conditions. The most investigated cortical area is the prefrontal cortex, since reported to play an important role in age-related compensatory mechanism and neurofunctional changes associated to these chronic neurological conditions. Interestingly, only 9 articles applied a longitudinal approach.

CONCLUSION

The results indicate that fNIRS is mainly employed for the cross-sectional characterization of the clinical phenotypes of these pathologies, whereas data on its utility for longitudinal monitoring as surrogate biomarkers of disease progression and rehabilitation effects are promising but still lacking.

Functional Near-Infrared Spectroscopy in the Study of Speech and Language Impairment Across the Life Span: A Systematic Review

Purpose Functional brain imaging is playing an increasingly important role in the diagnosis and treatment of communication disorders, yet many populations and settings are incompatible with functional magnetic resonance imaging and other commonly used techniques. We conducted a systematic review of neuroimaging studies using functional near-infrared spectroscopy (fNIRS) with individuals with speech or language impairment across the life span. We aimed to answer the following question: To what extent has fNIRS been used to investigate the neural correlates of speech-language impairment? Method This systematic review was preregistered with PROSPERO, the international prospective register of systematic reviews (CRD42019136464). We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol for preferred reporting items for systematic reviews. The database searches were conducted between February and March of 2019 with the following search terms: (a) fNIRS or functional near-infrared spectroscopy or NIRS or near-infrared spectroscopy, (b) speech or language, and (c) disorder or impairment or delay. Results We found 34 fNIRS studies that involved individuals with speech or language impairment across nine categories: (a) autism spectrum disorders; (b) developmental speech and language disorders; (c) cochlear implantation and deafness; (d) dementia, dementia of the Alzheimer's type, and mild cognitive impairment; (e) locked-in syndrome; (f) neurologic speech disorders/dysarthria; (g) stroke/aphasia; (h) stuttering; and (i) traumatic brain injury. Conclusions Though it is not without inherent challenges, fNIRS may have advantages over other neuroimaging techniques in the areas of speech and language impairment. fNIRS has clinical applications that may lead to improved early and differential diagnosis, increase our understanding of response to treatment, improve neuroprosthetic functioning, and advance neurofeedback.

Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain-Computer Interfaces

Ear-electroencephalography (EEG) is a promising tool for practical brain-computer interface (BCI) applications because it is more unobtrusive, comfortable, and mobile than a typical scalp-EEG system. However, an ear-EEG has a natural constraint of electrode location (e.g., limited in or around the ear) for acquiring informative brain signals sufficiently. Achieving reliable performance of ear-EEG in specific BCI paradigms that do not utilize brain signals on the temporal lobe around the ear is difficult. For example, steady-state visual evoked potentials (SSVEPs), which are mainly generated in the occipital area, have a significantly attenuated and distorted amplitude in ear-EEG. Therefore, preserving the high level of decoding accuracy is challenging and essential for SSVEP BCI based on ear-EEG. In this paper, we first investigate linear and nonlinear regression methods to increase the decoding accuracy of ear-EEG regarding SSVEP paradigm by utilizing the estimated target EEG signals on the occipital area. Then, we investigate an ensemble method to consider the prediction variability of the regression methods. Finally, we propose an error correction regression (ECR) framework to reduce the prediction errors by adding an additional nonlinear regression process (i.e., kernel ridge regression). We evaluate the ECR framework in terms of single session, session-to-session transfer, and subject-transfer decoding. We also validate the online decoding ability of the proposed framework with a short-time window size. The average accuracies are observed to be 91.11±9.14%, 90.52±8.67%, 86.96±12.13%, and 78.79±12.59%. This paper demonstrates that SSVEP BCI based on ear-EEG can achieve reliable performance with the proposed ECR framework.

The Potential of Functional Near-Infrared Spectroscopy-Based Neurofeedback-A Systematic Review and Recommendations for Best Practice

Background: The effects of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI)-neurofeedback on brain activation and behaviors have been studied extensively in the past. More recently, researchers have begun to investigate the effects of functional near-infrared spectroscopy-based neurofeedback (fNIRS-neurofeedback). FNIRS is a functional neuroimaging technique based on brain hemodynamics, which is easy to use, portable, inexpensive, and has reduced sensitivity to movement artifacts.

Method: We provide the first systematic review and database of fNIRS-neurofeedback studies, synthesizing findings from 22 peer-reviewed studies (including a total of N = 441 participants; 337 healthy, 104 patients). We (1) give a comprehensive overview of how fNIRS-neurofeedback training protocols were implemented, (2) review the online signal-processing methods used, (3) evaluate the quality of studies using pre-set methodological and reporting quality criteria and also present statistical sensitivity/power analyses, (4) investigate the effectiveness of fNIRS-neurofeedback in modulating brain activation, and (5) review its effectiveness in changing behavior in healthy and pathological populations.

Results and discussion: (1–2) Published studies are heterogeneous (e.g., neurofeedback targets, investigated populations, applied training protocols, and methods). (3) Large randomized controlled trials are still lacking. In view of the novelty of the field, the quality of the published studies is moderate. We identified room for improvement in reporting important information and statistical power to detect realistic effects. (4) Several studies show that people can regulate hemodynamic signals from cortical brain regions with fNIRS-neurofeedback and (5) these studies indicate the feasibility of modulating motor control and prefrontal brain functioning in healthy participants and ameliorating symptoms in clinical populations (stroke, ADHD, autism, and social anxiety). However, valid conclusions about specificity or potential clinical utility are premature.

Conclusion: Due to the advantages of practicability and relatively low cost, fNIRS-neurofeedback might provide a suitable and powerful alternative to EEG and fMRI neurofeedback and has great potential for clinical translation of neurofeedback. Together with more rigorous research and reporting practices, further methodological improvements may lead to a more solid understanding of fNIRS-neurofeedback. Future research will benefit from exploiting the advantages of fNIRS, which offers unique opportunities for neurofeedback research.

Random Subspace Ensemble Learning for Functional Near-Infrared Spectroscopy Brain-Computer Interfaces

The feasibility of the random subspace ensemble learning method was explored to improve the performance of functional near-infrared spectroscopy-based brain-computer interfaces (fNIRS-BCIs). Feature vectors have been constructed using the temporal characteristics of concentration changes in fNIRS chromophores such as mean, slope, and variance to implement fNIRS-BCIs systems. The mean and slope, which are the most popular features in fNIRS-BCIs, were adopted. Linear support vector machine and linear discriminant analysis were employed, respectively, as a single strong learner and multiple weak learners. All features in every channel and available time window were employed to train the strong learner, and the feature subsets were selected at random to train multiple weak learners. It was determined that random subspace ensemble learning is beneficial to enhance the performance of fNIRS-BCIs.

Hemodynamic responses during standing and sitting activities: a study toward fNIRS-BCI

In this paper, we utilized functional near-infrared spectroscopy (fNIRS) technology to examine the hemodynamic responses in the motor cortex for two conditions, namely standing and sitting tasks. Nine subjects performed five trials of standing and sitting (SAS) tasks with both real movements and imagery thinking of SAS. A group level of statistical parametric mapping (SPM) analysis during these tasks showed bilateral activation of oxy-hemoglobin for both real movements and imagery experiments. Interestingly, the SPM analysis clearly revealed that the sitting tasks induced a higher oxy-hemoglobin level activation compared to the standing task. Remarkably, this finding is persistent across the 22 measured channels at the individual and group levels for both experiments. Furthermore, six features were extracted from pre-processed HbO signals and the performance of four different classifiers was examined in order to test the viability of using SAS tasks in future fNIRS-brain-computer interface (fNIRS-BCI) systems. In particular, two features-combination tests revealed that the signal slope with signal variance represents one of the three best two-combined features for its consistency in providing high accuracy results for both real and imagery experiments. This study shows the potential of implementing such tasks into the fNIRS-BCI system. In the future, the results of this work could pave the way towards the application of fNIRS-BCI in lower limb rehabilitation.

fNIRS Complexity Analysis for the Assessment of Motor Imagery and Mental Arithmetic Tasks

Conventional methods for analyzing functional near-infrared spectroscopy (fNIRS) signals primarily focus on characterizing linear dynamics of the underlying metabolic processes. Nevertheless, linear analysis may underrepresent the true physiological processes that fully characterizes the complex and nonlinear metabolic activity sustaining brain function. Although there have been recent attempts to characterize nonlinearities in fNIRS signals in various experimental protocols, to our knowledge there has yet to be a study that evaluates the utility of complex characterizations of fNIRS in comparison to standard methods, such as the mean value of hemoglobin. Thus, the aim of this study was to investigate the entropy of hemoglobin concentration time series obtained from fNIRS signals and perform a comparitive analysis with standard mean hemoglobin analysis of functional activation. Publicly available data from 29 subjects performing motor imagery and mental arithmetics tasks were exploited for the purpose of this study. The experimental results show that entropy analysis on fNIRS signals may potentially uncover meaningful activation areas that enrich and complement the set identified through a traditional linear analysis.

Design of a wearable four-channel near-infrared spectroscopy system for the measurement of brain hemodynamic responses

Objectives This work describes the design and development of a four-channel near-infrared spectroscopy system to detect the oxygenated and deoxygenated hemoglobin concentration changes in the brain during various motor tasks. Methods The system uses light-emitting diodes corresponding to two wavelengths of 760 nm and 850 nm sensitive to deoxygenated and oxygenated hemoglobin concentration changes, respectively. The response is detected using a photodetector with an integrated transimpedance amplifier. The system is designed with four channels for functional near-infrared spectroscopy (fNIRS) signals acquisition. Two experiments were conducted to demonstrate the ability of the system to detect the changes in hemodynamic responses of different tasks. In the first experiment, the hemodynamic changes during motor execution and imagery of right- and left-fist clenching tasks were acquired by the developed system and validated against a standard multichannel NIRS system. In another experiment, the fNIRS signals during rest and motor execution of right-fist clenching task were acquired using the system and classified. Results The results demonstrate the ability of the designed system to detect the brain hemodynamic changes during various tasks. Also, the activation patterns obtained by the developed system with a minimum number of channels are on par with those obtained by the commercial system. Conclusions The developed four-channel NIRS system is user-friendly and has been designed with inexpensive components, unlike the commercially available NIRS instruments that are cumbersome and expensive.

Complexity Analysis on Functional-Near Infrared Spectroscopy Time Series: a Preliminary Study on Mental Arithmetic

It is well known that physiological systems show complex and nonlinear behaviours. In spite of that, functional near-infrared spectroscopy (fNIRS) is usually analyzed in the time and frequency domains with the assumption that metabolic activity is generated from a linear system. To leverage the full information provided by fNIRS signals, in this study we investigate topological entropy in fNIRS series collected from 10 healthy subjects during mental mental arithmetic task. While sample entropy and fuzzy entropy were used to estimate time series irregularity, distribution entropy was used to estimate time series complexity. Our findings show that entropy estimates may provide complementary characterization of fNIRS dynamics with respect to reference time domain measurements. This finding paves the way to further investigate functional activation in fNIRS in different case studies using nonlinear and complexity system theory.

Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface

Cognitive workload is one of the widely invoked human factors in the areas of human-machine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Real-time four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.

Cortical Tasks-Based Optimal Filter Selection: An fNIRS Study

Functional near-infrared spectroscopy (fNIRS) is one of the latest noninvasive brain function measuring technique that has been used for the purpose of brain-computer interfacing (BCI). In this paper, we compare and analyze the effect of six most commonly used filtering techniques (i.e., Gaussian, Butterworth, Kalman, hemodynamic response filter (hrf), Wiener, and finite impulse response) on classification accuracies of fNIRS-BCI. To conclude with the best optimal filter for a specific cortical task owing to a specific cortical region, we divided our experimental tasks according to the three main cortical regions: prefrontal, motor, and visual cortex. Three different experiments were performed for prefrontal and motor execution tasks while one for visual stimuli. The tasks performed for prefrontal include rest (R) vs mental arithmetic (MA), R vs object rotation (OB), and OB vs MA. Similarly, for motor execution, R vs left finger tapping (LFT), R vs right finger tapping (RFT), and LFT vs RFT. Likewise, for the visual cortex, R vs visual stimuli (VS) task. These experiments were performed for ten trials with five subjects. For consistency among extracted data, six statistical features were evaluated using oxygenated hemoglobin, namely, slope, mean, peak, kurtosis, skewness, and variance. Combination of these six features was used to classify data by the nonlinear support vector machine (SVM). The classification accuracies obtained from SVM by using hrf and Gaussian were significantly higher for R vs MA, R vs OB, R vs RFT, and R vs VS and Wiener filter for OB vs MA. Similarly, for R vs LFT and LFT vs RFT, hrf was found to be significant p<0.05. These results show the feasibility of using hrf for effective removal of noises from fNIRS data.

Detection of Mild Cognitive Impairment Using Convolutional Neural Network: Temporal-Feature Maps of Functional Near-Infrared Spectroscopy

Mild cognitive impairment (MCI) is the clinical precursor of Alzheimer's disease (AD), which is considered the most common neurodegenerative disease in the elderly. Some MCI patients tend to remain stable over time and do not evolve to AD. It is essential to diagnose MCI in its early stages and provide timely treatment to the patient. In this study, we propose a neuroimaging approach to identify MCI using a deep learning method and functional near-infrared spectroscopy (fNIRS). For this purpose, fifteen MCI subjects and nine healthy controls (HCs) were asked to perform three mental tasks: N-back, Stroop, and verbal fluency (VF) tasks. Besides examining the oxygenated hemoglobin changes (ΔHbO) in the region of interest, ΔHbO maps at 13 specific time points (i.e., 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, and 65 s) during the tasks and seven temporal feature maps (i.e., two types of mean, three types of slope, kurtosis, and skewness) in the prefrontal cortex were investigated. A four-layer convolutional neural network (CNN) was applied to identify the subjects into either MCI or HC, individually, after training the CNN model with ΔHbO maps and temporal feature maps above. Finally, we used the 5-fold cross-validation approach to evaluate the performance of the CNN. The results of temporal feature maps exhibited high classification accuracies: The average accuracies for the N-back task, Stroop task, and VFT, respectively, were 89.46, 87.80, and 90.37%. Notably, the highest accuracy of 98.61% was achieved from the ΔHbO slope map during 20-60 s interval of N-back tasks. Our results indicate that the fNIRS imaging approach based on temporal feature maps is a promising diagnostic method for early detection of MCI and can be used as a tool for clinical doctors to identify MCI from their patients.

Hemodynamics Analysis of Patients With Mild Cognitive Impairment During Working Memory Tasks

Diagnosis of dementia in early stage is important to prevent progression of dementia in the aging society. Mild cognitive impairment (MCI) denotes an early stage of Alzheimer disease (AD). In this paper, we aim to classify MCI patients from healthy controls (HC) during working memory tasks using functional near-infrared spectroscopy (fNIRS). To achieve this objective, t-values and correlation coefficients are calculated to find the region of interest (ROI) channels and brain connectivity. From the ROI channels averaged over subjects, features (mean and slope) of hemodynamic responses were extracted for classification. Extracted features were labelled as two classes and classified via two classifiers, linear discriminant analysis (LDA) and support vector machine (SVM). The classification accuracies were 73.08 % with LDA and 71.15 % with SVM. The results show that there are significant differences in the hemodynamic responses (HR) between MCI patients and healthy controls. Therefore, these results suggest a possibility of using fNIRS as a diagnostic tool for MCI patients.

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

Augmented reality (AR) enhances the user's environment by projecting virtual objects into the real world in real-time. Brain-computer interfaces (BCIs) are systems that enable users to control external devices with their brain signals. BCIs can exploit AR technology to interact with the physical and virtual world and to explore new ways of displaying feedback. This is important for users to perceive and regulate their brain activity or shape their communication intentions while operating in the physical world. In this study, twelve healthy participants were introduced to and asked to choose between two motor-imagery tasks: mental drawing and interacting with a virtual cube. Participants first performed a functional localizer run, which was used to select a single fNIRS channel for decoding their intentions in eight subsequent choice-encoding runs. In each run participants were asked to select one choice of a six-item list. A rotating AR cube was displayed on a computer screen as the main stimulus, where each face of the cube was presented for 6 s and represented one choice of the six-item list. For five consecutive trials, participants were instructed to perform the motor-imagery task when the face of the cube that represented their choice was facing them (therewith temporally encoding the selected choice). In the end of each run, participants were provided with the decoded choice based on a joint analysis of all five trials. If the decoded choice was incorrect, an active error-correction procedure was applied by the participant. The choice list provided in each run was based on the decoded choice of the previous run. The experimental design allowed participants to navigate twice through a virtual menu that consisted of four levels if all choices were correctly decoded. Here we demonstrate for the first time that by using AR feedback and flexible choice encoding in form of search trees, we can increase the degrees of freedom of a BCI system. We also show that participants can successfully navigate through a nested menu and achieve a mean accuracy of 74% using a single motor-imagery task and a single fNIRS channel.

Mental Workload Classification Method Based on EEG Independent Component Features

Excessive mental workload will reduce work efficiency, but low mental workload will cause a waste of human resources. It is very significant to study the mental workload status of operators. The existing mental workload classification method is based on electroencephalogram (EEG) features, and its classification accuracy is often low because the channel signals recorded by the EEG electrodes are a group of mixed brain signals, which are similar to multi-source mixed speech signals. It is not wise to directly analyze the mixed signals in order to distinguish the feature of EEG signals. In this study, we propose a mental workload classification method based on EEG independent components (ICs) features, which borrows from the blind source separation (BSS) idea of mixed speech signals. This presented method uses independent component analysis (ICA) to obtain pure signals, i.e., ICs. The energy features of ICs are directly extracted for classifying the mental workload, since this method directly uses ICs energy features for feature extraction. Compared with the existing solution, the proposed method can obtain better classification results. The presented method might provide a way to realize a fast, accurate, and automatic mental workload classification.

[Recognition of three different imagined movement of the right foot based on functional near-infrared spectroscopy]

Brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) is a new-type human-computer interaction technique. To explore the separability of fNIRS signals in different motor imageries on the single limb, the study measured the fNIRS signals of 15 subjects (amateur football fans) during three different motor imageries of the right foot (passing, stopping and shooting). And the correlation coefficient of the HbO signal during different motor imageries was extracted as features for the input of a three-classification model based on support vector machines. The results found that the classification accuracy of the three motor imageries of the right foot was 78.89%±6.161%. The classification accuracy of the two-classification of motor imageries of the right foot, that is, passing and stopping, passing and shooting, and stopping and shooting was 85.17%±4.768%, 82.33%±6.011%, and 89.33%±6.713%, respectively. The results demonstrate that the fNIRS of different motor imageries of the single limb is separable, which is expected to add new control commands to fNIRS-BCI and also provide a new option for rehabilitation training and control peripherals for unilateral stroke patients. Besides, the study also confirms that the correlation coefficient can be used as an effective feature to classify different motor imageries.

Resting-State NIRS-EEG in Unresponsive Patients with Acute Brain Injury: A Proof-of-Concept Study

BACKGROUND

Neurovascular-based imaging techniques such as functional MRI (fMRI) may reveal signs of consciousness in clinically unresponsive patients but are often subject to logistical challenges in the intensive care unit (ICU). Near-infrared spectroscopy (NIRS) is another neurovascular imaging technique but low cost, can be performed serially at the bedside, and may be combined with electroencephalography (EEG), which are important advantages compared to fMRI. Combined NIRS-EEG, however, has never been evaluated for the assessment of neurovascular coupling and consciousness in acute brain injury.

METHODS

We explored resting-state oscillations in eight-channel NIRS oxyhemoglobin and eight-channel EEG band-power signals to assess neurovascular coupling, the prerequisite for neurovascular-based imaging detection of consciousness, in patients with acute brain injury in the ICU (n = 9). Conscious neurological patients from step-down units and wards served as controls (n = 14). Unsupervised adaptive mixture-independent component analysis (AMICA) was used to correlate NIRS-EEG data with levels of consciousness and clinical outcome.

RESULTS

Neurovascular coupling between NIRS oxyhemoglobin (0.07-0.13 Hz) and EEG band-power (1-12 Hz) signals at frontal areas was sensitive and prognostic to changing consciousness levels. AMICA revealed a mixture of five models from EEG data, with the relative probabilities of these models reflecting levels of consciousness over multiple days, although the accuracy was less than 85%. However, when combined with two channels of bilateral frontal neurovascular coupling, weighted k-nearest neighbor classification of AMICA probabilities distinguished unresponsive patients from conscious controls with > 90% accuracy (positive predictive value 93%, false discovery rate 7%) and, additionally, identified patients who subsequently failed to recover consciousness with > 99% accuracy.

DISCUSSION

We suggest that NIRS-EEG for monitoring of acute brain injury in the ICU is worthy of further exploration. Normalization of neurovascular coupling may herald recovery of consciousness after acute brain injury.

Functional Near-Infrared Spectroscopy for the Classification of Motor-Related Brain Activity on the Sensor-Level

Sensor-level human brain activity is studied during real and imaginary motor execution using functional near-infrared spectroscopy (fNIRS). Blood oxygenation and deoxygenation spatial dynamics exhibit pronounced hemispheric lateralization when performing motor tasks with the left and right hands. This fact allowed us to reveal biomarkers of hemodynamical response of the motor cortex on the motor execution, and use them for designing a sensing method for classification of the type of movement. The recognition accuracy of real movements is close to 100%, while the classification accuracy of imaginary movements is lower but quite high (at the level of 90%). The advantage of the proposed method is its ability to classify real and imaginary movements with sufficiently high efficiency without the need for recalculating parameters. The proposed system can serve as a sensor of motor activity to be used for neurorehabilitation after severe brain injuries, including traumas and strokes.

Brain-Based Binary Communication Using Spatiotemporal Features of fNIRS Responses

“Locked-in” patients lose their ability to communicate naturally due to motor system dysfunction. Brain-computer interfacing offers a solution for their inability to communicate by enabling motor-independent communication. Straightforward and convenient in-session communication is essential in clinical environments. The present study introduces a functional near-infrared spectroscopy (fNIRS)-based binary communication paradigm that requires limited preparation time and merely nine optodes. Eighteen healthy participants performed two mental imagery tasks, mental drawing and spatial navigation, to answer yes/no questions during one of two auditorily cued time windows. Each of the six questions was answered five times, resulting in five trials per answer. This communication paradigm thus combines both spatial (two different mental imagery tasks, here mental drawing for “yes” and spatial navigation for “no”) and temporal (distinct time windows for encoding a “yes” and “no” answer) fNIRS signal features for information encoding. Participants’ answers were decoded in simulated real-time using general linear model analysis. Joint analysis of all five encoding trials resulted in an average accuracy of 66.67 and 58.33% using the oxygenated (HbO) and deoxygenated (HbR) hemoglobin signal respectively. For half of the participants, an accuracy of 83.33% or higher was reached using either the HbO signal or the HbR signal. For four participants, effective communication with 100% accuracy was achieved using either the HbO or HbR signal. An explorative analysis investigated the differentiability of the two mental tasks based solely on spatial fNIRS signal features. Using multivariate pattern analysis (MVPA) group single-trial accuracies of 58.33% (using 20 training trials per task) and 60.56% (using 40 training trials per task) could be obtained. Combining the five trials per run using a majority voting approach heightened these MVPA accuracies to 62.04 and 75%. Additionally, an fNIRS suitability questionnaire capturing participants’ physical features was administered to explore its predictive value for evaluating general data quality. Obtained questionnaire scores correlated significantly (r = -0.499) with the signal-to-noise of the raw light intensities. While more work is needed to further increase decoding accuracy, this study shows the potential of answer encoding using spatiotemporal fNIRS signal features or spatial fNIRS signal features only.

On fractality of functional near-infrared spectroscopy signals: analysis and applications

Significance: The human brain is a highly complex system with nonlinear, dynamic behavior. A majority of brain imaging studies employing functional near-infrared spectroscopy (fNIRS), however, have considered only the spatial domain and have ignored the temporal properties of fNIRS recordings. Methods capable of revealing nonlinearities in fNIRS recordings can provide new insights about how the brain functions. Aim: The temporal characteristics of fNIRS signals are explored by comprehensively investigating their fractal properties. Approach: Fractality of fNIRS signals is analyzed using scaled windowed variance (SWV), as well as using visibility graph (VG), a method which converts a given time series into a graph. Additionally, the fractality of fNIRS signals obtained under resting-state and task-based conditions is compared, and the application of fractality in differentiating brain states is demonstrated for the first time via various classification approaches. Results: Results from SWV analysis show the existence of high fractality in fNIRS recordings. It is shown that differences in the temporal characteristics of fNIRS signals related to task-based and resting-state conditions can be revealed via the VGs constructed for each case. Conclusions: fNIRS recordings, regardless of the experimental conditions, exhibit high fractality. Furthermore, VG-based metrics can be employed to differentiate rest and task-execution brain states.

Performance assessment of high-density diffuse optical topography regarding source-detector array topology

Recent advances in optical neuroimaging systems as a functional interface enhance our understanding of neuronal activity in the brain. High density diffuse optical topography (HD-DOT) uses multi-distance overlapped channels to improve the spatial resolution of images comparable to functional magnetic resonance imaging (fMRI). The topology of the source and detector (SD) array directly impacts the quality of the hemodynamic reconstruction in HD-DOT imaging modality. In this work, the effect of different SD configurations on the quality of cerebral hemodynamic recovery is investigated by presenting a simulation setup based on the analytical approach. Given that the SD arrangement determines the elements of the Jacobian matrix, we conclude that the more individual components in this matrix, the better the retrieval quality. The results demonstrate that the multi-distance multi-directional (MDMD) arrangement produces more unique elements in the Jacobian array. Consequently, the inverse problem can accurately retrieve the brain activity of diffuse optical topography data.

Differences in Net Information Flow and Dynamic Connectivity Metrics Between Physically Active and Inactive Subjects Measured by Functional Near-Infrared Spectroscopy (fNIRS) During a Fatiguing Handgrip Task

Twenty-three young adults (4 Females, 25.13 ± 3.72 years) performed an intermittent maximal handgrip force task using their dominant hand for 20 min (3.5 s squeeze/6.5 s release, 120 blocks) with concurrent cortical activity imaging by functional Near-Infrared Spectroscopy (fNRIS; OMM-3000, Shimadzu Corp., 111 channels). Subjects were grouped as physically active (n = 10) or inactive (n = 12) based on a questionnaire (active-exercise at least four times a week, inactive- exercise less than two times a week). We explored how motor task fatigue affected the vasomotion-induced oscillations in ΔHbO as measured by fNIRS at each hemodynamic frequency band: endothelial component (0.003–0.02 Hz) associated to microvascular activity, neurogenic component (0.02–0.04 Hz) related to intrinsic neuronal activity, and myogenic component (0.04–0.15 Hz) linked to activity of smooth muscles of arterioles. To help understand how these three neurovascular regulatory mechanisms relate to handgrip task performance we quantified several dynamic fNIRS metrics, including directional phase transfer entropy (dPTE), a computationally efficient and data-driven method used as a marker of information flow between cortical regions, and directional connectivity (DC), a means to compute directionality of information flow between two cortical regions. The relationship between static functional connectivity (SFC) and functional connectivity variability (FCV) was also explored to understand their mutual dependence for each frequency band in the context of handgrip performance as fatigued increased. Our findings ultimately showed differences between subject groups across all fNIRS metrics and hemodynamic frequency bands. These findings imply that physical activity modulates neurovascular control mechanisms at the endogenic, neurogenic, and myogenic frequency bands resulting in delayed fatigue onset and enhanced performance. The dynamic cortical network metrics quantified in this work for young, healthy subjects provides baseline measurements to guide future work on older individuals and persons with impaired cardiovascular health.

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interface Using Regularized Linear Discriminant Analysis Ensemble Classifier Based on Bootstrap Aggregating

Ensemble classifiers have been proven to result in better classification accuracy than that of a single strong learner in many machine learning studies. Although many studies on electroencephalography-brain-computer interface (BCI) used ensemble classifiers to enhance the BCI performance, ensemble classifiers have hardly been employed for near-infrared spectroscopy (NIRS)-BCIs. In addition, since there has not been any systematic and comparative study, the efficacy of ensemble classifiers for NIRS-BCIs remains unknown. In this study, four NIRS-BCI datasets were employed to evaluate the efficacy of linear discriminant analysis ensemble classifiers based on the bootstrap aggregating. From the analysis results, significant (or marginally significant) increases in the bitrate as well as the classification accuracy were found for all four NIRS-BCI datasets employed in this study. Moreover, significant bitrate improvements were found in two of the four datasets.

The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience

The past few decades have seen a rapid increase in the use of functional near-infrared spectroscopy (fNIRS) in cognitive neuroscience. This fast growth is due to the several advances that fNIRS offers over the other neuroimaging modalities such as functional magnetic resonance imaging and electroencephalography/magnetoencephalography. In particular, fNIRS is harmless, tolerant to bodily movements, and highly portable, being suitable for all possible participant populations, from newborns to the elderly and experimental settings, both inside and outside the laboratory. In this review we aim to provide a comprehensive and state-of-the-art review of fNIRS basics, technical developments, and applications. In particular, we discuss some of the open challenges and the potential of fNIRS for cognitive neuroscience research, with a particular focus on neuroimaging in naturalistic environments and social cognitive neuroscience.

Reduction of Onset Delay in Functional Near-Infrared Spectroscopy: Prediction of HbO/HbR Signals

An intrinsic problem when using hemodynamic responses for the brain-machine interface is the slow nature of the physiological process. In this paper, a novel method that estimates the oxyhemoglobin changes caused by neuronal activations is proposed and validated. In monitoring the time responses of blood-oxygen-level-dependent signals with functional near-infrared spectroscopy (fNIRS), the early trajectories of both oxy- and deoxy-hemoglobins in their phase space are scrutinized. Furthermore, to reduce the detection time, a prediction method based upon a kernel-based recursive least squares (KRLS) algorithm is implemented. In validating the proposed approach, the fNIRS signals of finger tapping tasks measured from the left motor cortex are examined. The results show that the KRLS algorithm using the Gaussian kernel yields the best fitting for both ΔHbO (i.e., 87.5%) and ΔHbR (i.e., 85.2%) at q = 15 steps ahead (i.e., 1.63 s ahead at a sampling frequency of 9.19 Hz). This concludes that a neuronal activation can be concluded in about 0.1 s with fNIRS using prediction, which enables an almost real-time practice if combined with EEG.

Using the General Linear Model to Improve Performance in fNIRS Single Trial Analysis and Classification: A Perspective

Within a decade, single trial analysis of functional Near Infrared Spectroscopy (fNIRS) signals has gained significant momentum, and fNIRS joined the set of modalities frequently used for active and passive Brain Computer Interfaces (BCI). A great variety of methods for feature extraction and classification have been explored using state-of-the-art Machine Learning methods. In contrast, signal preprocessing and cleaning pipelines for fNIRS often follow simple recipes and so far rarely incorporate the available state-of-the-art in adjacent fields. In neuroscience, where fMRI and fNIRS are established neuroimaging tools, evoked hemodynamic brain activity is typically estimated across multiple trials using a General Linear Model (GLM). With the help of the GLM, subject, channel, and task specific evoked hemodynamic responses are estimated, and the evoked brain activity is more robustly separated from systemic physiological interference using independent measures of nuisance regressors, such as short-separation fNIRS measurements. When correctly applied in single trial analysis, e.g., in BCI, this approach can significantly enhance contrast to noise ratio of the brain signal, improve feature separability and ultimately lead to better classification accuracy. In this manuscript, we provide a brief introduction into the GLM and show how to incorporate it into a typical BCI preprocessing pipeline and cross-validation. Using a resting state fNIRS data set augmented with synthetic hemodynamic responses that provide ground truth brain activity, we compare the quality of commonly used fNIRS features for BCI that are extracted from (1) conventionally preprocessed signals, and (2) signals preprocessed with the GLM and physiological nuisance regressors. We show that the GLM-based approach can provide better single trial estimates of brain activity as well as a new feature type, i.e., the weight of the individual and channel-specific hemodynamic response function (HRF) regressor. The improved estimates yield features with higher separability, that significantly enhance accuracy in a binary classification task when compared to conventional preprocessing—on average +7.4% across subjects and feature types. We propose to adapt this well-established approach from neuroscience to the domain of single-trial analysis and preprocessing wherever the classification of evoked brain activity is of concern, for instance in BCI.