Determining Optimal Feature-Combination for LDA Classification of Functional Near-Infrared Spectroscopy Signals in Brain-Computer Interface Application

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism

Brain Computer Interface (BCI) is a highly promising human-computer interaction method that can utilize brain signals to control external devices. BCI based on functional near-infrared spectroscopy (fNIRS) is considered a relatively new and promising paradigm. fNIRS is a technique of measuring functional changes in cerebral hemodynamics. It detects changes in the hemodynamic activity of the cerebral cortex by measuring oxyhemoglobin and deoxyhemoglobin (HbR) concentrations and inversely predicts the neural activity of the brain. At the present time, Deep learning (DL) methods have not been widely used in fNIRS decoding, and there are fewer studies considering both spatial and temporal dimensions for fNIRS classification. To solve these problems, we proposed an end-to-end hybrid neural network for feature extraction of fNIRS. The method utilizes a spatial-temporal convolutional layer for automatic extraction of temporally valid information and uses a spatial attention mechanism to extract spatially localized information. A temporal convolutional network (TCN) is used to further utilize the temporal information of fNIRS before the fully connected layer. We validated our approach on a publicly available dataset including 29 subjects, including left-hand and right-hand motor imagery (MI), mental arithmetic (MA), and a baseline task. The results show that the method has few training parameters and high accuracy, providing a meaningful reference for BCI development.

fNIRS-EEG BCIs for Motor Rehabilitation: A Review

Motor impairment has a profound impact on a significant number of individuals, leading to a substantial demand for rehabilitation services. Through brain-computer interfaces (BCIs), people with severe motor disabilities could have improved communication with others and control appropriately designed robotic prosthetics, so as to (at least partially) restore their motor abilities. BCI plays a pivotal role in promoting smoother communication and interactions between individuals with motor impairments and others. Moreover, they enable the direct control of assistive devices through brain signals. In particular, their most significant potential lies in the realm of motor rehabilitation, where BCIs can offer real-time feedback to assist users in their training and continuously monitor the brain's state throughout the entire rehabilitation process. Hybridization of different brain-sensing modalities, especially functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), has shown great potential in the creation of BCIs for rehabilitating the motor-impaired populations. EEG, as a well-established methodology, can be combined with fNIRS to compensate for the inherent disadvantages and achieve higher temporal and spatial resolution. This paper reviews the recent works in hybrid fNIRS-EEG BCIs for motor rehabilitation, emphasizing the methodologies that utilized motor imagery. An overview of the BCI system and its key components was introduced, followed by an introduction to various devices, strengths and weaknesses of different signal processing techniques, and applications in neuroscience and clinical contexts. The review concludes by discussing the possible challenges and opportunities for future development.

Driving drowsiness detection using spectral signatures of EEG-based neurophysiology

Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks.

Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics.

Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen’s kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods.

Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.

A Hybrid GCN and Filter-Based Framework for Channel and Feature Selection: An fNIRS-BCI Study

In this study, a channel and feature selection methodology is devised for brain-computer interface (BCI) applications using functional near-infrared spectroscopy (fNIRS). A graph convolutional network (GCN) is employed to select the appropriate and correlated fNIRS channels. Furthermore, in the feature extraction phase, the performance of two filter-based feature selection algorithms, (i) the minimum redundancy maximum relevance (mRMR) and (ii) ReliefF, is investigated. The five most commonly used temporal statistical features (i.e., mean, slope, maximum, skewness, and kurtosis) are used, whereas the conventional support vector machine (SVM) is utilized as a classifier for training and testing. The proposed methodology is validated using an available online dataset of motor imagery (left- and right-hand), mental arithmetic, and baseline tasks. First, the efficacy of the proposed methodology is shown for two-class BCI applications (i.e., left- vs. right-hand motor imagery and mental arithmetic vs. baseline). Second, the proposed framework is applied to four-class BCI applications (i.e., left- vs. right-hand motor imagery vs. mental arithmetic vs. baseline). The results show that the number of appropriate channels and features was significantly reduced, resulting in a significant increase in classification accuracy for both two-class and four-class BCI applications, respectively. Furthermore, both mRMR (i.e., 87.8% for motor imagery, 87.1% for mental arithmetic, and 78.7% for four-class) and ReliefF (i.e., 90.7% for motor imagery, 93.7% for mental arithmetic, and 81.6% for four-class) yielded high average classification accuracy $\left(p<0.05\right)$ . However, the results of the ReliefF algorithm are more stable and significant.

Explainable artificial intelligence model to predict brain states from fNIRS signals

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode's output onto the input variables for fNIRS signals is described here. Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model's output in terms of the model's input. Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations. Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI

Decoding neural responses from multimodal information sources, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), has the transformative potential to advance hybrid brain-computer interfaces (hBCIs). However, existing modest performance improvement of hBCIs might be attributed to the lack of computational frameworks that exploit complementary synergistic properties in multimodal features. This study proposes a multimodal data fusion framework to represent and decode synergistic multimodal motor imagery (MI) neural responses. We hypothesize that exploiting EEG nonlinear dynamics adds a new informative dimension to the commonly combined EEG-fNIRS features and will ultimately increase the synergy between EEG and fNIRS features toward an enhanced hBCI. The EEG nonlinear dynamics were quantified by extracting graph-based recurrence quantification analysis (RQA) features to complement the commonly used spectral features for an enhanced multimodal configuration when combined with fNIRS. The high-dimensional multimodal features were further given to a feature selection algorithm relying on the least absolute shrinkage and selection operator (LASSO) for fused feature selection. Linear support vector machine (SVM) was then used to evaluate the framework. The mean hybrid classification performance improved by up to 15% and 4% compared to the unimodal EEG and fNIRS, respectively. The proposed graph-based framework substantially increased the contribution of EEG features for hBCI classification from 28.16% up to 52.9% when introduced the nonlinear dynamics and improved the performance by approximately 2%. These findings suggest that graph-based nonlinear dynamics can increase the synergy between EEG and fNIRS features for an enhanced MI response representation that is not dominated by a single modality.

Migraine classification by machine learning with functional near-infrared spectroscopy during the mental arithmetic task

Migraine is a common and complex neurovascular disorder. Clinically, the diagnosis of migraine mainly relies on scales, but the degree of pain is too subjective to be a reliable indicator. It is even more difficult to diagnose the medication-overuse headache, which can only be evaluated by whether the symptom is improved after the medication adjustment. Therefore, an objective migraine classification system to assist doctors in making a more accurate diagnosis is needed. In this research, 13 healthy subjects (HC), 9 chronic migraine subjects (CM), and 12 medication-overuse headache subjects (MOH) were measured by functional near-infrared spectroscopy (fNIRS) to observe the change of the hemoglobin in the prefrontal cortex (PFC) during the mental arithmetic task (MAT). Our model shows the sensitivity and specificity of CM are 100% and 75%, and that of MOH is 75% and 100%.The results of the classification of the three groups prove that fNIRS combines with machine learning is feasible for the migraine classification.

LASSO Homotopy-Based Sparse Representation Classification for fNIRS-BCI

Brain-computer interface (BCI) systems based on functional near-infrared spectroscopy (fNIRS) have been used as a way of facilitating communication between the brain and peripheral devices. The BCI provides an option to improve the walking pattern of people with poor walking dysfunction, by applying a rehabilitation process. A state-of-the-art step-wise BCI system includes data acquisition, pre-processing, channel selection, feature extraction, and classification. In fNIRS-based BCI (fNIRS-BCI), channel selection plays a vital role in enhancing the classification accuracy of the BCI problem. In this study, the concentration of blood oxygenation (HbO) in a resting state and in a walking state was used to decode the walking activity and the resting state of the subject, using channel selection by Least Absolute Shrinkage and Selection Operator (LASSO) homotopy-based sparse representation classification. The fNIRS signals of nine subjects were collected from the left hemisphere of the primary motor cortex. The subjects performed the task of walking on a treadmill for 10 s, followed by a 20 s rest. Appropriate filters were applied to the collected signals to remove motion artifacts and physiological noises. LASSO homotopy-based sparse representation was used to select the most significant channels, and then classification was performed to identify walking and resting states. For comparison, the statistical spatial features of mean, peak, variance, and skewness, and their combination, were used for classification. The classification results after channel selection were then compared with the classification based on the extracted features. The classifiers used for both methods were linear discrimination analysis (LDA), support vector machine (SVM), and logistic regression (LR). The study found that LASSO homotopy-based sparse representation classification successfully discriminated between the walking and resting states, with a better average classification accuracy (p < 0.016) of 91.32%. This research provides a step forward in improving the classification accuracy of fNIRS-BCI systems. The proposed methodology may also be used for rehabilitation purposes, such as controlling wheelchairs and prostheses, as well as an active rehabilitation training technique for patients with motor dysfunction.

Analyzing Classification Performance of fNIRS-BCI for Gait Rehabilitation Using Deep Neural Networks

This research presents a brain-computer interface (BCI) framework for brain signal classification using deep learning (DL) and machine learning (ML) approaches on functional near-infrared spectroscopy (fNIRS) signals. fNIRS signals of motor execution for walking and rest tasks are acquired from the primary motor cortex in the brain's left hemisphere for nine subjects. DL algorithms, including convolutional neural networks (CNNs), long short-term memory (LSTM), and bidirectional LSTM (Bi-LSTM) are used to achieve average classification accuracies of 88.50%, 84.24%, and 85.13%, respectively. For comparison purposes, three conventional ML algorithms, support vector machine (SVM), k-nearest neighbor (k-NN), and linear discriminant analysis (LDA) are also used for classification, resulting in average classification accuracies of 73.91%, 74.24%, and 65.85%, respectively. This study successfully demonstrates that the enhanced performance of fNIRS-BCI can be achieved in terms of classification accuracy using DL approaches compared to conventional ML approaches. Furthermore, the control commands generated by these classifiers can be used to initiate and stop the gait cycle of the lower limb exoskeleton for gait rehabilitation.

Acupuncture enhances brain function in patients with mild cognitive impairment: evidence from a functional-near infrared spectroscopy study

Mild cognitive impairment (MCI) is a precursor to Alzheimer’s disease. It is imperative to develop a proper treatment for this neurological disease in the aging society. This observational study investigated the effects of acupuncture therapy on MCI patients. Eleven healthy individuals and eleven MCI patients were recruited for this study. Oxy- and deoxy-hemoglobin signals in the prefrontal cortex during working-memory tasks were monitored using functional near-infrared spectroscopy. Before acupuncture treatment, working-memory experiments were conducted for healthy control (HC) and MCI groups (MCI-0), followed by 24 sessions of acupuncture for the MCI group. The acupuncture sessions were initially carried out for 6 weeks (two sessions per week), after which experiments were performed again on the MCI group (MCI-1). This was followed by another set of acupuncture sessions that also lasted for 6 weeks, after which the experiments were repeated on the MCI group (MCI-2). Statistical analyses of the signals and classifications based on activation maps as well as temporal features were performed. The highest classification accuracies obtained using binary connectivity maps were 85.7% HC vs. MCI-0, 69.5% HC vs. MCI-1, and 61.69% HC vs. MCI-2. The classification accuracies using the temporal features mean from 5 seconds to 28 seconds and maximum (i.e, max(5:28 seconds)) values were 60.6% HC vs. MCI-0, 56.9% HC vs. MCI-1, and 56.4% HC vs. MCI-2. The results reveal that there was a change in the temporal characteristics of the hemodynamic response of MCI patients due to acupuncture. This was reflected by a reduction in the classification accuracy after the therapy, indicating that the patients’ brain responses improved and became comparable to those of healthy subjects. A similar trend was reflected in the classification using the image feature. These results indicate that acupuncture can be used for the treatment of MCI patients.

Recent Advances and Current Trends in Brain-Computer Interface (BCI) Research and Their Applications

Brain-Computer Interface (BCI) provides direct communication between the brain and an external device. BCI systems have become a trendy field of research in recent years. These systems can be used in a variety of applications to help both disabled and healthy people. Concerning significant BCI progress, we may assume that these systems are not very far from real-world applications. This review has taken into account current trends in BCI research. In this survey, one hundred most cited articles from the WOS database were selected over the last four years. This survey is divided into several sectors. These sectors are Medicine, Communication and Control, Entertainment, and Other BCI applications. The application area, recording method, signal acquisition types, and countries of origin have been identified in each article. This survey provides an overview of the BCI articles published from 2016 to 2020 and their current trends and advances in different application areas.

Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e., optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky–Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.

Most favorable stimulation duration in the sensorimotor cortex for fNIRS-based BCI

One of the primary objectives of the brain-computer interface (BCI) is to obtain a command with higher classification accuracy within the shortest possible time duration. Therefore, this study evaluates several stimulation durations to propose a duration that can yield the highest classification accuracy. Furthermore, this study aims to address the inherent delay in the hemodynamic responses (HRs) for the command generation time. To this end, HRs in the sensorimotor cortex were evaluated for the functional near-infrared spectroscopy (fNIRS)-based BCI. To evoke brain activity, right-hand-index finger poking and tapping tasks were used. In this study, six different stimulation durations (i.e., 1, 3, 5, 7, 10, and 15 s) were tested on 10 healthy male subjects. Upon stimulation, different temporal features and multiple time windows were utilized to extract temporal features. The extracted features were then classified using linear discriminant analysis. The classification results using the main HR showed that a 5 s stimulation duration could yield the highest classification accuracy, i.e., 74%, with a combination of the mean and maximum value features. However, the results were not significantly different from the classification accuracy obtained using the 15 s stimulation. To further validate the results, a classification using the initial dip was performed. The results obtained endorsed the finding with an average classification accuracy of 73.5% using the features of minimum peak and skewness in the 5 s window. The results based on classification using the initial dip for 5 s were significantly different from all other tested stimulation durations (p < 0.05) for all feature combinations. Moreover, from the visual inspection of the HRs, it is observed that the initial dip occurred as soon as the task started, but the main HR had a delay of more than 2 s. Another interesting finding is that impulsive stimulation in the sensorimotor cortex can result in the generation of a clearer initial dip phenomenon. The results reveal that the command for the fNIRS-based BCI can be generated using the 5 s stimulation duration. In conclusion, the use of the initial dip can reduce the time taken for the generation of commands and can be used to achieve a higher classification accuracy for the fNIRS-BCI within a 5 s task duration rather than relying on longer durations.

Crossing time windows optimization based on mutual information for hybrid BCI

Hybrid EEG-fNIRS brain-computer interface (HBCI) is widely employed to enhance BCI performance. EEG and fNIRS signals are combined to increase the dimensionality of the information. Time windows are used to select EEG and fNIRS singles synchronously. However, it ignores that specific modal signals have their own characteristics, when the task is stimulated, the information between the modalities will mismatch at the moment, which has a significant impact on the classification performance. Here we propose a novel crossing time windows optimization for mental arithmetic (MA) based BCI. The EEG and fNIRS signals were segmented separately by sliding time windows. Then crossing time windows (CTW) were combined with each one segment from EEG and fNIRS selected independently. Furthermore, EEG and fNIRS features were extracted using Filter Bank Common Spatial Pattern (FBCSP) and statistical methods from each sample. Mutual information was calculated for FBCSP and statistical features to characterize the discrimination of crossing time windows, and the optimal window would be selected based on the largest mutual information. Finally, a sparse structured framework of Fisher Lasso feature selection (FLFS) was designed to select the joint features, and conventional Linear Discriminant Analysis (LDA) was employed to perform classification. We used proposed method for a MA dataset. The classification accuracy of the proposed method is 92.52 ± 5.38% and higher than other methods, which shows the rationality and superiority of the proposed method.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Motor Training Using Mental Workload (MWL) With an Assistive Soft Exoskeleton System: A Functional Near-Infrared Spectroscopy (fNIRS) Study for Brain-Machine Interface (BMI)

Mental workload is a neuroergonomic human factor, which is widely used in planning a system's safety and areas like brain-machine interface (BMI), neurofeedback, and assistive technologies. Robotic prosthetics methodologies are employed for assisting hemiplegic patients in performing routine activities. Assistive technologies' design and operation are required to have an easy interface with the brain with fewer protocols, in an attempt to optimize mobility and autonomy. The possible answer to these design questions may lie in neuroergonomics coupled with BMI systems. In this study, two human factors are addressed: designing a lightweight wearable robotic exoskeleton hand that is used to assist the potential stroke patients with an integrated portable brain interface using mental workload (MWL) signals acquired with portable functional near-infrared spectroscopy (fNIRS) system. The system may generate command signals for operating a wearable robotic exoskeleton hand using two-state MWL signals. The fNIRS system is used to record optical signals in the form of change in concentration of oxy and deoxygenated hemoglobin (HbO and HbR) from the pre-frontal cortex (PFC) region of the brain. Fifteen participants participated in this study and were given hand-grasping tasks. Two-state MWL signals acquired from the PFC region of the participant's brain are segregated using machine learning classifier-support vector machines (SVM) to utilize in operating a robotic exoskeleton hand. The maximum classification accuracy is 91.31%, using a combination of mean-slope features with an average information transfer rate (ITR) of 1.43. These results show the feasibility of a two-state MWL (fNIRS-based) robotic exoskeleton hand (BMI system) for hemiplegic patients assisting in the physical grasping tasks.

Optimizing Mental Workload Estimation by Detecting Baseline State Using Vector Phase Analysis Approach

Improper baseline return from the previous task-evoked hemodynamic response (HR) can contribute to a large variation in the subsequent HR, affecting the estimation of mental workload in brain-computer interface systems. In this study, we proposed a method using vector phase analysis to detect the baseline state as being optimal or suboptimal. We hypothesize that selecting neuronal-related HR as observed in the optimal-baseline blocks can lead to an improvement in estimating mental workload. Oxygenated and deoxygenated hemoglobin concentration changes were integrated as parts of the vector phase. The proposed method was applied to a block-design functional near-infrared spectroscopy dataset (total blocks = 1384), measured on 24 subjects performing multiple difficulty levels of mental arithmetic task. Significant differences in hemodynamic signal change were observed between the optimal- and suboptimal-baseline blocks detected using the proposed method. This supports the effectiveness of the proposed method in detecting baseline state for better estimation of mental workload. The results further highlight the need of customized recovery duration. In short, the proposed method offers a practical approach to detect task-evoked signals, without the need of extra probes.

Quantitative Assessment of Resting-State for Mild Cognitive Impairment Detection: A Functional Near-Infrared Spectroscopy and Deep Learning Approach

BACKGROUND

Mild cognitive impairment (MCI) is considered a prodromal stage of Alzheimer's disease. Early diagnosis of MCI can allow for treatment to improve cognitive function and reduce modifiable risk factors.

OBJECTIVE

This study aims to investigate the feasibility of individual MCI detection from healthy control (HC) using a minimum duration of resting-state functional near-infrared spectroscopy (fNIRS) signals.

METHODS

In this study, nine different measurement durations (i.e., 30, 60, 90, 120, 150, 180, 210, 240, and 270 s) were evaluated for MCI detection via the graph theory analysis and traditional machine learning approach, such as linear discriminant analysis, support vector machine, and K-nearest neighbor algorithms. Moreover, feature representation- and classification-based transfer learning (TL) methods were applied to identify MCI from HC through the input of connectivity maps with 30 and 90 s duration.

RESULTS

There was no significant difference among the nine various time windows in the machine learning and graph theory analysis. The feature representation-based TL showed improved accuracy in both 30 and 90 s cases (i.e., 30 s: 81.27% and 90 s: 76.73%). Notably, the classification-based TL method achieved the highest accuracy of 95.81% using the pre-trained convolutional neural network (CNN) model with the 30 s interval functional connectivity map input.

CONCLUSION

The results indicate that a 30 s measurement of the resting-state with fNIRS could be used to detect MCI. Moreover, the combination of neuroimaging (e.g., functional connectivity maps) and deep learning methods (e.g., CNN and TL) can be considered as novel biomarkers for clinical computer-assisted MCI diagnosis.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

Subject-independent decoding of affective states using functional near-infrared spectroscopy

Affective decoding is the inference of human emotional states using brain signal measurements. This approach is crucial to develop new therapeutic approaches for psychiatric rehabilitation, such as affective neurofeedback protocols. To reduce the training duration and optimize the clinical outputs, an ideal clinical neurofeedback could be trained using data from an independent group of volunteers before being used by new patients. Here, we investigated if this subject-independent design of affective decoding can be achieved using functional near-infrared spectroscopy (fNIRS) signals from frontal and occipital areas. For this purpose, a linear discriminant analysis classifier was first trained in a dataset (49 participants, 24.65±3.23 years) and then tested in a completely independent one (20 participants, 24.00±3.92 years). Significant balanced accuracies between classes were found for positive vs. negative (64.50 ± 12.03%, p<0.01) and negative vs. neutral (68.25 ± 12.97%, p<0.01) affective states discrimination during a reactive block consisting in viewing affective-loaded images. For an active block, in which volunteers were instructed to recollect personal affective experiences, significant accuracy was found for positive vs. neutral affect classification (71.25 ± 18.02%, p<0.01). In this last case, only three fNIRS channels were enough to discriminate between neutral and positive affective states. Although more research is needed, for example focusing on better combinations of features and classifiers, our results highlight fNIRS as a possible technique for subject-independent affective decoding, reaching significant classification accuracies of emotional states using only a few but biologically relevant features.

Enhancing Classification Performance of fNIRS-BCI by Identifying Cortically Active Channels Using the z-Score Method

A state-of-the-art brain–computer interface (BCI) system includes brain signal acquisition, noise removal, channel selection, feature extraction, classification, and an application interface. In functional near-infrared spectroscopy-based BCI (fNIRS-BCI) channel selection may enhance classification performance by identifying suitable brain regions that contain brain activity. In this study, the z-score method for channel selection is proposed to improve fNIRS-BCI performance. The proposed method uses cross-correlation to match the similarity between desired and recorded brain activity signals, followed by forming a vector of each channel’s correlation coefficients’ maximum values. After that, the z-score is calculated for each value of that vector. A channel is selected based on a positive z-score value. The proposed method is applied to an open-access dataset containing mental arithmetic (MA) and motor imagery (MI) tasks for twenty-nine subjects. The proposed method is compared with the conventional t-value method and with no channel selected, i.e., using all channels. The z-score method yielded significantly improved (p < 0.0167) classification accuracies of 87.2 ± 7.0%, 88.4 ± 6.2%, and 88.1 ± 6.9% for left motor imagery (LMI) vs. rest, right motor imagery (RMI) vs. rest, and mental arithmetic (MA) vs. rest, respectively. The proposed method is also validated on an open-access database of 17 subjects, containing right-hand finger tapping (RFT), left-hand finger tapping (LFT), and dominant side foot tapping (FT) tasks.The study shows an enhanced performance of the z-score method over the t-value method as an advancement in efforts to improve state-of-the-art fNIRS-BCI systems’ performance.

Performance Improvement for Detecting Brain Function Using fNIRS: A Multi-Distance Probe Configuration With PPL Method

To improve the spatial resolution of imaging and get more effective brain function information, a multi-distance probe configuration with three distances (28.2, 40, and 44.7 mm) and 52 channels is designed. At the same time, a data conversion method of modified Beer–Lambert law (MBLL) with partial pathlength (PPL) is proposed. In the experiment, three kinds of tasks, grip of left hand, grip of right hand, and rest, are performed with eight healthy subjects. First, with a typical single-distance probe configuration (30 mm, 24 channels), the feasibility of the proposed MBLL with PPL is preliminarily validated. Further, the characteristic of the proposed method is evaluated with the multi-distance probe configuration. Compared with MBLL with differential pathlength factor (DPF), the proposed MBLL with PPL is able to acquire more obvious concentration change and can achieve higher classification accuracy of the three tasks. Then, with the proposed method, the performance of the multi-distance probe configuration is discussed. Results show that, compared with a single distance, the combination of the three distances has better spatial resolution and could explore more accurate brain activation information. Besides, the classification accuracy of the three tasks obtained with the combination of three distances is higher than that of any combination of two distances. Also, with the combination of the three distances, the two-class classification between different tasks is carried out. Both theory and experimental results demonstrate that, using multi-distance probe configuration and the MBLL with PPL method, the performance of brain function detected by NIRS can be improved.

Subject-Specific feature selection for near infrared spectroscopy based brain-computer interfaces

BACKGROUND AND OBJECTIVE

Brain-computer interfaces (BCIs) enable people to control an external device by analyzing the brain's neural activity. Functional near-infrared spectroscopy (fNIRS), which is an emerging optical imaging technique, is frequently used in non-invasive BCIs. Determining the subject-specific features is an important concern in enhancing the classification accuracy as well as reducing the complexity of fNIRS based BCI systems. In this study, the effectiveness of subject-specific feature selection on classification accuracy of fNIRS signals is examined.

METHODS

In order to determine the subject-specific optimal feature subsets, stepwise regression analysis based on sequential feature selection (SWR-SFS) and ReliefF methods were employed. Feature selection is applied on time-domain features of fNIRS signals such as mean, slope, peak, skewness and kurtosis values of signals. Linear discriminant analysis, k nearest neighborhood and support vector machines are employed to evaluate the performance of the selected feature subsets. The proposed techniques are validated on benchmark motor imagery (MI) and mental arithmetic (MA) based fNIRS datasets collected from 29 healthy subjects.

RESULTS

Both SWR-SFS and reliefF feature selection methods have significantly improved the classification accuracy. However, the best results (88.67% (HbR) and 86.43% (HbO) for MA dataset and 77.01% (HbR) and 71.32% (HbO) for MI dataset) were achieved using SWR-SFS while feature selection provided extremely high feature reduction rates (89.50% (HbR) and 93.99% (HbO) for MA dataset and 94.04% (HbR) and 97.73% (HbO) for MI dataset).

CONCLUSIONS

The results of the study indicate that employing feature selection improves both MA and MI-based fNIRS signals classification performance significantly.

A Unified Analytical Framework With Multiple fNIRS Features for Mental Workload Assessment in the Prefrontal Cortex

Knowing the actual level of mental workload is important to ensure the efficacy of brain-computer interface (BCI) based cognitive training. Extracting signals from limited area of a brain region might not reveal the actual information. In this study, a functional near-infrared spectroscopy (fNIRS) device equipped with multi-channel and multi-distance measurement capability was employed for the development of an analytical framework to assess mental workload in the prefrontal cortex (PFC). In addition to the conventional features, e.g. hemodynamic slope, we introduced a new feature - deep contribution ratio which is the proportion of cerebral hemodynamics to the fNIRS signals. Multiple sets of features were examined by a simple logical operator to suppress the false detection rate in identifying the activated channels. Using the number of activated channels as input to a linear support vector machine (SVM), the performance of the proposed analytical framework was assessed in classifying three levels of mental workload. The best set of features involves the combination of hemodynamic slope and deep contribution ratio, where the identified number of activated channels returned an average accuracy of 80.6% in predicting mental workload, compared to a single conventional feature (accuracy: 59.8%). This suggests the feasibility of the proposed analytical framework with multiple features as a means towards a more accurate assessment of mental workload in fNIRS-based BCI applications.

Hemodynamic responses during standing and sitting activities: a study toward fNIRS-BCI

In this paper, we utilized functional near-infrared spectroscopy (fNIRS) technology to examine the hemodynamic responses in the motor cortex for two conditions, namely standing and sitting tasks. Nine subjects performed five trials of standing and sitting (SAS) tasks with both real movements and imagery thinking of SAS. A group level of statistical parametric mapping (SPM) analysis during these tasks showed bilateral activation of oxy-hemoglobin for both real movements and imagery experiments. Interestingly, the SPM analysis clearly revealed that the sitting tasks induced a higher oxy-hemoglobin level activation compared to the standing task. Remarkably, this finding is persistent across the 22 measured channels at the individual and group levels for both experiments. Furthermore, six features were extracted from pre-processed HbO signals and the performance of four different classifiers was examined in order to test the viability of using SAS tasks in future fNIRS-brain-computer interface (fNIRS-BCI) systems. In particular, two features-combination tests revealed that the signal slope with signal variance represents one of the three best two-combined features for its consistency in providing high accuracy results for both real and imagery experiments. This study shows the potential of implementing such tasks into the fNIRS-BCI system. In the future, the results of this work could pave the way towards the application of fNIRS-BCI in lower limb rehabilitation.

Cortical Tasks-Based Optimal Filter Selection: An fNIRS Study

Functional near-infrared spectroscopy (fNIRS) is one of the latest noninvasive brain function measuring technique that has been used for the purpose of brain-computer interfacing (BCI). In this paper, we compare and analyze the effect of six most commonly used filtering techniques (i.e., Gaussian, Butterworth, Kalman, hemodynamic response filter (hrf), Wiener, and finite impulse response) on classification accuracies of fNIRS-BCI. To conclude with the best optimal filter for a specific cortical task owing to a specific cortical region, we divided our experimental tasks according to the three main cortical regions: prefrontal, motor, and visual cortex. Three different experiments were performed for prefrontal and motor execution tasks while one for visual stimuli. The tasks performed for prefrontal include rest (R) vs mental arithmetic (MA), R vs object rotation (OB), and OB vs MA. Similarly, for motor execution, R vs left finger tapping (LFT), R vs right finger tapping (RFT), and LFT vs RFT. Likewise, for the visual cortex, R vs visual stimuli (VS) task. These experiments were performed for ten trials with five subjects. For consistency among extracted data, six statistical features were evaluated using oxygenated hemoglobin, namely, slope, mean, peak, kurtosis, skewness, and variance. Combination of these six features was used to classify data by the nonlinear support vector machine (SVM). The classification accuracies obtained from SVM by using hrf and Gaussian were significantly higher for R vs MA, R vs OB, R vs RFT, and R vs VS and Wiener filter for OB vs MA. Similarly, for R vs LFT and LFT vs RFT, hrf was found to be significant p<0.05. These results show the feasibility of using hrf for effective removal of noises from fNIRS data.

Detection of Mild Cognitive Impairment Using Convolutional Neural Network: Temporal-Feature Maps of Functional Near-Infrared Spectroscopy

Mild cognitive impairment (MCI) is the clinical precursor of Alzheimer's disease (AD), which is considered the most common neurodegenerative disease in the elderly. Some MCI patients tend to remain stable over time and do not evolve to AD. It is essential to diagnose MCI in its early stages and provide timely treatment to the patient. In this study, we propose a neuroimaging approach to identify MCI using a deep learning method and functional near-infrared spectroscopy (fNIRS). For this purpose, fifteen MCI subjects and nine healthy controls (HCs) were asked to perform three mental tasks: N-back, Stroop, and verbal fluency (VF) tasks. Besides examining the oxygenated hemoglobin changes (ΔHbO) in the region of interest, ΔHbO maps at 13 specific time points (i.e., 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, and 65 s) during the tasks and seven temporal feature maps (i.e., two types of mean, three types of slope, kurtosis, and skewness) in the prefrontal cortex were investigated. A four-layer convolutional neural network (CNN) was applied to identify the subjects into either MCI or HC, individually, after training the CNN model with ΔHbO maps and temporal feature maps above. Finally, we used the 5-fold cross-validation approach to evaluate the performance of the CNN. The results of temporal feature maps exhibited high classification accuracies: The average accuracies for the N-back task, Stroop task, and VFT, respectively, were 89.46, 87.80, and 90.37%. Notably, the highest accuracy of 98.61% was achieved from the ΔHbO slope map during 20-60 s interval of N-back tasks. Our results indicate that the fNIRS imaging approach based on temporal feature maps is a promising diagnostic method for early detection of MCI and can be used as a tool for clinical doctors to identify MCI from their patients.

Hemodynamics Analysis of Patients With Mild Cognitive Impairment During Working Memory Tasks

Diagnosis of dementia in early stage is important to prevent progression of dementia in the aging society. Mild cognitive impairment (MCI) denotes an early stage of Alzheimer disease (AD). In this paper, we aim to classify MCI patients from healthy controls (HC) during working memory tasks using functional near-infrared spectroscopy (fNIRS). To achieve this objective, t-values and correlation coefficients are calculated to find the region of interest (ROI) channels and brain connectivity. From the ROI channels averaged over subjects, features (mean and slope) of hemodynamic responses were extracted for classification. Extracted features were labelled as two classes and classified via two classifiers, linear discriminant analysis (LDA) and support vector machine (SVM). The classification accuracies were 73.08 % with LDA and 71.15 % with SVM. The results show that there are significant differences in the hemodynamic responses (HR) between MCI patients and healthy controls. Therefore, these results suggest a possibility of using fNIRS as a diagnostic tool for MCI patients.

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interface Using Regularized Linear Discriminant Analysis Ensemble Classifier Based on Bootstrap Aggregating

Ensemble classifiers have been proven to result in better classification accuracy than that of a single strong learner in many machine learning studies. Although many studies on electroencephalography-brain-computer interface (BCI) used ensemble classifiers to enhance the BCI performance, ensemble classifiers have hardly been employed for near-infrared spectroscopy (NIRS)-BCIs. In addition, since there has not been any systematic and comparative study, the efficacy of ensemble classifiers for NIRS-BCIs remains unknown. In this study, four NIRS-BCI datasets were employed to evaluate the efficacy of linear discriminant analysis ensemble classifiers based on the bootstrap aggregating. From the analysis results, significant (or marginally significant) increases in the bitrate as well as the classification accuracy were found for all four NIRS-BCI datasets employed in this study. Moreover, significant bitrate improvements were found in two of the four datasets.

Reduction of Onset Delay in Functional Near-Infrared Spectroscopy: Prediction of HbO/HbR Signals

An intrinsic problem when using hemodynamic responses for the brain-machine interface is the slow nature of the physiological process. In this paper, a novel method that estimates the oxyhemoglobin changes caused by neuronal activations is proposed and validated. In monitoring the time responses of blood-oxygen-level-dependent signals with functional near-infrared spectroscopy (fNIRS), the early trajectories of both oxy- and deoxy-hemoglobins in their phase space are scrutinized. Furthermore, to reduce the detection time, a prediction method based upon a kernel-based recursive least squares (KRLS) algorithm is implemented. In validating the proposed approach, the fNIRS signals of finger tapping tasks measured from the left motor cortex are examined. The results show that the KRLS algorithm using the Gaussian kernel yields the best fitting for both ΔHbO (i.e., 87.5%) and ΔHbR (i.e., 85.2%) at q = 15 steps ahead (i.e., 1.63 s ahead at a sampling frequency of 9.19 Hz). This concludes that a neuronal activation can be concluded in about 0.1 s with fNIRS using prediction, which enables an almost real-time practice if combined with EEG.

Open-Access fNIRS Dataset for Classification of Unilateral Finger- and Foot-Tapping

Numerous open-access electroencephalography (EEG) datasets have been released and widely employed by EEG researchers. However, not many functional near-infrared spectroscopy (fNIRS) datasets are publicly available. More fNIRS datasets need to be freely accessible in order to facilitate fNIRS studies. Toward this end, we introduce an open-access fNIRS dataset for three-class classification. The concentration changes of oxygenated and reduced hemoglobin were measured, while 30 volunteers repeated each of the three types of overt movements (i.e., left- and right-hand unilateral complex finger-tapping, foot-tapping) for 25 times. The ternary support vector machine (SVM) classification accuracy obtained using leave-one-out cross-validation was estimated at 70.4% ± 18.4% on average. A total of 21 out of 30 volunteers scored a superior binary SVM classification accuracy (left-hand vs. right-hand finger-tapping) of over 80.0%. We believe that the introduced fNIRS dataset can facilitate future fNIRS studies.

Assessing the brain 'on the line': An ecologically-valid assessment of the impact of repetitive assembly line work on hemodynamic response and fine motor control using fNIRS

To investigate neural correlates of repetitive assembly tasks in ecologically-valid empirical settings, this study measured bilateral prefrontal (PFC) and motor activations when participants performed a carburetor assembly task using functional near-infrared spectroscopy (fNIRS). Participants worked for one hour at a typical (low-) pace and at an accelerated high-pace. Before and after the task, a test was conducted to assess motion stability and fine motor control. The behavioral data revealed decreased motion stability after the assembly work in both conditions, with a significantly higher reduction after the high-pace task. The fNIRS data also revealed reduced activations in bilateral prefrontal and motor regions in both conditions over time. However, the low-pace task led to significantly greater activity decreases compared with the high-pace. Activity decrease in prefrontal and motor regions within the low pace also significantly related to minimal motion stability impairment, suggesting that the brain activation decreases in this and, potentially, findings of higher alpha in past repetitive-task studies using EEG, may be a result of not fatigue but worker adaptation or increasing efficiency.

Evaluation of Neural Degeneration Biomarkers in the Prefrontal Cortex for Early Identification of Patients With Mild Cognitive Impairment: An fNIRS Study

Mild cognitive impairment (MCI), a condition characterizing poor cognition, is associated with aging and depicts early symptoms of severe cognitive impairment, known as Alzheimer's disease (AD). Meanwhile, early detection of MCI can prevent progression to AD. A great deal of research has been performed in the past decade on MCI detection. However, availability of biomarkers for MCI detection requires greater attention. In our study, we evaluated putative and reliable biomarkers for diagnosing MCI by performing different mental tasks (i.e., N-back task, Stroop task, and verbal fluency task) using functional near-infrared spectroscopy (fNIRS) signals on a group of 15 MCI patients and 9 healthy control (HC). The 15 digital biomarkers (i.e., five means, seven slopes, peak, skewness, and kurtosis) and two image biomarkers (t-map, correlation map) in the prefrontal cortex (PFC) (i.e., left PFC, middle PFC, and right PFC) between the MCI and HC groups were investigated by the statistical analysis, linear discriminant analysis (LDA), and convolutional neural network (CNN) individually. The results reveal that the statistical analysis using digital biomarkers (with a p-value < 0.05) could not distinguish the MCI patients from the HC over 60% accuracy. Therefore, the current statistical analysis needs to be improved to be used for diagnosing the MCI patients. The best accuracy with LDA was 76.67% with the N-back and Stroop tasks. However, the CNN classification results trained by image biomarkers showed a high accuracy. In particular, the CNN results trained via t-maps revealed the best accuracy (90.62%) with the N-back task, whereas the CNN result trained by the correlation maps was 85.58% with the N-back task. Also, the results illustrated that investigating the sub-regions (i.e., right, middle, left) of the PFC for detecting MCI would be better than examining the whole PFC. The t-map (or/and the correlation map) is conclusively recommended as an image biomarker for early detection of AD. The combination of CNN and image biomarkers can provide a reliable clinical tool for diagnosing MCI patients.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

Early Detection of Alzheimer's Disease Using Non-invasive Near-Infrared Spectroscopy

Mild cognitive impairment (MCI) is a cognitive disorder characterized by memory impairment, wherein patients have an increased likelihood of developing Alzheimer’s disease (AD). The classification of MCI and different AD stages is therefore fundamental for understanding and treating the disease. This study aimed to comprehensively investigate the hemodynamic response patterns among various subject groups. Functional near-infrared spectroscopy (fNIRS) was employed to measure signals from the frontal and bilateral parietal cortices of healthy controls (n = 8), patients with MCI (n = 9), mild (n = 6), and moderate/severe AD (n = 7) during a digit verbal span task (DVST). The concentration changes of oxygenated hemoglobin (HbO) in various subject groups were thoroughly explored and tested. Result revealed that abnormal patterns of hemodynamic response were observed across all subject groups. Greater and steeper reductions in HbO concentration were consistently observed across all regions of interest (ROIs) as disease severity developed from MCI to moderate/severe AD. Furthermore, all the fNIRS-derived indexes were found to be significantly and positively correlated to the clinical scores in all ROIs (R ≥ 0.4, P < 0.05). These findings demonstrate the feasibility of utilizing fNIRS for the early detection of AD, suggesting that fNIRS-based approaches hold great promise for exploring the mechanisms underlying the progression of AD.

Existence of Initial Dip for BCI: An Illusion or Reality

A tight coupling between the neuronal activity and the cerebral blood flow (CBF) is the motivation of many hemodynamic response (HR)-based neuroimaging modalities. The increase in neuronal activity causes the increase in CBF that is indirectly measured by HR modalities. Upon functional stimulation, the HR is mainly categorized in three durations: (i) initial dip, (ii) conventional HR (i.e., positive increase in HR caused by an increase in the CBF), and (iii) undershoot. The initial dip is a change in oxygenation prior to any subsequent increase in CBF and spatially more specific to the site of neuronal activity. Despite additional evidence from various HR modalities on the presence of initial dip in human and animal species (i.e., cat, rat, and monkey); the existence/occurrence of an initial dip in HR is still under debate. This article reviews the existence and elusive nature of the initial dip duration of HR in intrinsic signal optical imaging (ISOI), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The advent of initial dip and its elusiveness factors in ISOI and fMRI studies are briefly discussed. Furthermore, the detection of initial dip and its role in brain-computer interface using fNIRS is examined in detail. The best possible application for the initial dip utilization and its future implications using fNIRS are provided.

Detecting Concealed Information with Fused Electroencephalography and Functional Near-infrared Spectroscopy

In this study, fused electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) techniques were utilized to examine the relationship between the ERP (event-related potential) component P300 and fNIRS hemodynamic signals for high-accuracy deception detection. During the performance of a modified concealed information test (CIT) task, a series of Chinese names were presented, which served as the target, irrelevant, or the probe stimuli for both the guilty and innocent groups. For participants in the guilty group, the probe stimulus was their individual name, whereas for the innocent group, the probe stimulus was one irrelevant name. In particular, data from concurrent fNIRS and ERP recordings were carefully inspected for participants from the two groups. Interestingly, we discovered that for the guilty group, the probe stimulus elicited significantly higher P300 amplitude at parietal site and also evoked significantly stronger oxyhemoglobin (HbO) concentration changes in the bilateral superior frontal gyrus and bilateral middle frontal gyrus than the irrelevant stimuli. However, this is not the case for the innocent group, in which participants exhibited no significant differences in both ERP and fNIRS measures between the probe and irrelevant stimuli. More importantly, our findings also demonstrated that the combined ERP and fNIRS feature was able to differentiate the guilty and innocent groups with enhanced sensitivity, in which AUC (the area under Receiver Operating Characteristic curve) is 0.94 for deception detection based on the combined indicator, much higher than that based on the ERP component P300 only (0.85) or HbO measure only (0.84).

Functional near-infrared spectroscopy-based affective neurofeedback: feedback effect, illiteracy phenomena, and whole-connectivity profiles

Background: Affective neurofeedback constitutes a suitable approach to control abnormal neural activities associated with psychiatric disorders and might consequently relief symptom severity. However, different aspects of neurofeedback remain unclear, such as its neural basis, the performance variation, the feedback effect, among others. Aim: First, we aimed to propose a functional near-infrared spectroscopy (fNIRS)-based affective neurofeedback based on the self-regulation of frontal and occipital networks. Second, we evaluated three different feedback approaches on performance: real, fixed, and random feedback. Third, we investigated different demographic, psychological, and physiological predictors of performance. Approach: Thirty-three healthy participants performed a task whereby an amorphous figure changed its shape according to the elicited affect (positive or neutral). During the task, the participants randomly received three different feedback approaches: real feedback, with no change of the classifier output; fixed feedback, keeping the feedback figure unmodified; and random feedback, where the classifier output was multiplied by an arbitrary value, causing a feedback different than expected by the subject. Then, we applied a multivariate comparison of the whole-connectivity profiles according to the affective states and feedback approaches, as well as during a pretask resting-state block, to predict performance. Results: Participants were able to control this feedback system with 70.00 % ± 24.43 % ( p < 0.01 ) of performance during the real feedback trials. No significant differences were found when comparing the average performances of the feedback approaches. However, the whole functional connectivity profiles presented significant Mahalanobis distances ( p ≪ 0.001 ) when comparing both affective states and all feedback approaches. Finally, task performance was positively correlated to the pretask resting-state whole functional connectivity ( r = 0.512 , p = 0.009 ). Conclusions: Our results suggest that fNIRS might be a feasible tool to develop a neurofeedback system based on the self-regulation of affective networks. This finding enables future investigations using an fNIRS-based affective neurofeedback in psychiatric populations. Furthermore, functional connectivity profiles proved to be a good predictor of performance and suggested an increased effort to maintain task control in the presence of feedback distractors.

Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Neuronal Activation Detection Using Vector Phase Analysis with Dual Threshold Circles: A Functional Near-Infrared Spectroscopy Study

In this paper, a new vector phase diagram differentiating the initial decreasing phase (i.e. initial dip) and the delayed hemodynamic response (HR) phase of oxy-hemoglobin changes ( Δ HbO) of functional near-infrared spectroscopy (fNIRS) is developed. The vector phase diagram displays the trajectories of Δ HbO and deoxy-hemoglobin changes ( Δ HbR), as orthogonal components, in the Δ HbO- Δ HbR polar coordinates. To determine the occurrence of an initial dip, dual threshold circles (an inner circle from the resting state, an outer circle from the peak values of the initial dip and the main HR) are incorporated into the phase diagram for making decisions. The proposed scheme is then applied to a brain-computer interface scheme, and its performance is evaluated in classifying two finger tapping tasks (right-hand thumb and little finger) from the left motor cortex. Three gamma functions are used to model the initial dip, the main HR, and the undershoot in generating the designed HR function. In classifying two tapping tasks, the signal mean and signal minimum values during 0-2.5 s, as features of initial dip, are used. The linear discriminant analysis was utilized as a classifier. The experimental results show that the active brain locations of the two tasks were quite distinctive ( p < 0.05 ), and moreover, spatially specific if using the initial dip map at 4 s in comparison to the map of HRs at 14 s. Also, the average classification accuracy was improved from 59% to 74.9% when using the phase diagram of dual threshold circles.

Shining a Light on Awareness: A Review of Functional Near-Infrared Spectroscopy for Prolonged Disorders of Consciousness

Qualitative clinical assessments of the recovery of awareness after severe brain injury require an assessor to differentiate purposeful behavior from spontaneous behavior. As many such behaviors are minimal and inconsistent, behavioral assessments are susceptible to diagnostic errors. Advanced neuroimaging tools can bypass behavioral responsiveness and reveal evidence of covert awareness and cognition within the brains of some patients, thus providing a means for more accurate diagnoses, more accurate prognoses, and, in some instances, facilitated communication. The majority of reports to date have employed the neuroimaging methods of functional magnetic resonance imaging, positron emission tomography, and electroencephalography (EEG). However, each neuroimaging method has its own advantages and disadvantages (e.g., signal resolution, accessibility, etc.). Here, we describe a burgeoning technique of non-invasive optical neuroimaging—functional near-infrared spectroscopy (fNIRS)—and review its potential to address the clinical challenges of prolonged disorders of consciousness. We also outline the potential for simultaneous EEG to complement the fNIRS signal and suggest the future directions of research that are required in order to realize its clinical potential.

Lie Detection Using fNIRS Monitoring of Inhibition-Related Brain Regions Discriminates Infrequent but not Frequent Liars

Functional near-infrared spectroscopy (fNIRS) was used to test whether monitoring inhibition-related brain regions is a feasible method for detecting both infrequent liars and frequent liars. Thirty-two participants were divided into two groups: the deceptive group (liars) and the non-deceptive group (ND group, innocents). All the participants were required to undergo a simulated interrogation by a computer. The participants from the deceptive group were instructed to tell a mix of lies and truths and those of the ND group were instructed always to tell the truth. Based on the number of deceptions, the participants of the deceptive group were further divided into a infrequently deceptive group (IFD group, infrequent liars) and a frequently deceptive group (FD group, frequent liars). The infrequent liars exhibited greater neural activities than the frequent liars and the innocents in the left middle frontal gyrus (MFG) when performing the deception detection tasks. While performing deception detection tasks, infrequent liars showed significantly greater neural activation in the left MFG than the baseline, but frequent liars and innocents did not exhibit this pattern of neural activation in any area of inhibition-related brain regions. The results of individual analysis showed an acceptable accuracy of detecting infrequent liars, but an unacceptable accuracy of detecting frequent liars. These results suggest that using fNIRS monitoring of inhibition-related brain regions is feasible for detecting infrequent liars, for whom deception may be more effortful and therefore more physiologically marked, but not frequent liars.

Robust Adaptive Synchronization of Ring Configured Uncertain Chaotic FitzHugh-Nagumo Neurons under Direction-Dependent Coupling

This paper exploits the dynamical modeling, behavior analysis, and synchronization of a network of four different FitzHugh–Nagumo (FHN) neurons with unknown parameters linked in a ring configuration under direction-dependent coupling. The main purpose is to investigate a robust adaptive control law for the synchronization of uncertain and perturbed neurons, communicating in a medium of bidirectional coupling. The neurons are assumed to be different and interconnected in a ring structure. The strength of the gap junctions is taken to be different for each link in the network, owing to the inter-neuronal coupling medium properties. Robust adaptive control mechanism based on Lyapunov stability analysis is employed and theoretical criteria are derived to realize the synchronization of the network of four FHN neurons in a ring form with unknown parameters under direction-dependent coupling and disturbances. The proposed scheme for synchronization of dissimilar neurons, under external electrical stimuli, coupled in a ring communication topology, having all parameters unknown, and subject to directional coupling medium and perturbations, is addressed for the first time as per our knowledge. To demonstrate the efficacy of the proposed strategy, simulation results are provided.

fNIRS-based Neurorobotic Interface for gait rehabilitation

BACKGROUND

In this paper, a novel functional near-infrared spectroscopy (fNIRS)-based brain-computer interface (BCI) framework for control of prosthetic legs and rehabilitation of patients suffering from locomotive disorders is presented.

METHODS

fNIRS signals are used to initiate and stop the gait cycle, while a nonlinear proportional derivative computed torque controller (PD-CTC) with gravity compensation is used to control the torques of hip and knee joints for minimization of position error. In the present study, the brain signals of walking intention and rest tasks were acquired from the left hemisphere's primary motor cortex for nine subjects. Thereafter, for removal of motion artifacts and physiological noises, the performances of six different filters (i.e. Kalman, Wiener, Gaussian, hemodynamic response filter (hrf), Band-pass, finite impulse response) were evaluated. Then, six different features were extracted from oxygenated hemoglobin signals, and their different combinations were used for classification. Also, the classification performances of five different classifiers (i.e. k-Nearest Neighbour, quadratic discriminant analysis, linear discriminant analysis (LDA), Naïve Bayes, support vector machine (SVM)) were tested.

RESULTS

The classification accuracies obtained from SVM using the hrf were significantly higher (p < 0.01) than those of the other classifier/ filter combinations. Those accuracies were 77.5, 72.5, 68.3, 74.2, 73.3, 80.8, 65, 76.7, and 86.7% for the nine subjects, respectively.

CONCLUSION

The control commands generated using the classifiers initiated and stopped the gait cycle of the prosthetic leg, the knee and hip torques of which were controlled using the PD-CTC to minimize the position error. The proposed scheme can be effectively used for neurofeedback training and rehabilitation of lower-limb amputees and paralyzed patients.