Enhancing classification accuracy of fNIRS-BCI using features acquired from vector-based phase analysis

Unleashing the potential of fNIRS with machine learning: classification of fine anatomical movements to empower future brain-computer interface

In this study, we explore the potential of using functional near-infrared spectroscopy (fNIRS) signals in conjunction with modern machine-learning techniques to classify specific anatomical movements to increase the number of control commands for a possible fNIRS-based brain-computer interface (BCI) applications. The study focuses on novel individual finger-tapping, a well-known task in fNIRS and fMRI studies, but limited to left/right or few fingers. Twenty-four right-handed participants performed the individual finger-tapping task. Data were recorded by using sixteen sources and detectors placed over the motor cortex according to the 10-10 international system. The event's average oxygenated Δ HbO and deoxygenated Δ HbR hemoglobin data were utilized as features to assess the performance of diverse machine learning (ML) models in a challenging multi-class classification setting. These methods include LDA, QDA, MNLR, XGBoost, and RF. A new DL-based model named "Hemo-Net" has been proposed which consists of multiple parallel convolution layers with different filters to extract the features. This paper aims to explore the efficacy of using fNRIS along with ML/DL methods in a multi-class classification task. Complex models like RF, XGBoost, and Hemo-Net produce relatively higher test set accuracy when compared to LDA, MNLR, and QDA. Hemo-Net has depicted a superior performance achieving the highest test set accuracy of 76%, however, in this work, we do not aim at improving the accuracies of models rather we are interested in exploring if fNIRS has the neural signatures to help modern ML/DL methods in multi-class classification which can lead to applications like brain-computer interfaces. Multi-class classification of fine anatomical movements, such as individual finger movements, is difficult to classify with fNIRS data. Traditional ML models like MNLR and LDA show inferior performance compared to the ensemble-based methods of RF and XGBoost. DL-based method Hemo-Net outperforms all methods evaluated in this study and demonstrates a promising future for fNIRS-based BCI applications.

fNIRS-based graph frequency analysis to identify mild cognitive impairment in Parkinson's disease

BACKGROUND

Early identification of mild cognitive impairment (MCI) is essential for its treatment and the prevention of dementia in Parkinson's disease (PD). Existing approaches are mostly based on neuropsychological assessments, while brain activation and connection have not been well considered.

NEW METHOD

This paper presents a neuroimaging-based graph frequency analysis method and the generated features to quantify the brain functional neurodegeneration and distinguish between PD-MCI patients and healthy controls. The Stroop color-word experiment was conducted with 20 PD-MCI patients and 34 healthy controls, and the brain activation was recorded with functional near-infrared spectroscopy (fNIRS). Then, the functional brain network was constructed based on Pearson's correlation coefficient calculation between every two fNIRS channels. Next, the functional brain network was represented as a graph and decomposed in the graph frequency domain through the graph Fourier transform (GFT) to obtain the eigenvector matrix. Total variation and weighted zero crossings of eigenvectors were defined and integrated to quantify functional interaction between brain regions and the spatial variability of the brain network in specific graph frequency ranges, respectively. After that, the features were employed in training a support vector machine (SVM) classifier.

RESULTS

The presented method achieved a classification accuracy of 0.833 and an F1 score of 0.877, significantly outperforming existing methods and features.

COMPARISON WITH EXISTING METHODS

Our method provided improved classification performance in the identification of PD-MCI.

CONCLUSION

The results suggest that the presented graph frequency analysis method well identify PD-MCI patients and the generated features promise functional brain biomarkers for PD-MCI diagnosis.

A global bibliometric and visualized analysis of gait analysis and artificial intelligence research from 1992 to 2022

Gait is an important basic function of human beings and an integral part of life. Many mental and physical abnormalities can cause noticeable differences in a person's gait. Abnormal gait can lead to serious consequences such as falls, limited mobility and reduced life satisfaction. Gait analysis, which includes joint kinematics, kinetics, and dynamic Electromyography (EMG) data, is now recognized as a clinically useful tool that can provide both quantifiable and qualitative information on performance to aid in treatment planning and evaluate its outcome. With the assistance of new artificial intelligence (AI) technology, the traditional medical environment has undergone great changes. AI has the potential to reshape medicine, making gait analysis more accurate, efficient and accessible. In this study, we analyzed basic information about gait analysis and AI articles that met inclusion criteria in the WoS Core Collection database from 1992-2022, and the VosViewer software was used for web visualization and keyword analysis. Through bibliometric and visual analysis, this article systematically introduces the research status of gait analysis and AI. We introduce the application of artificial intelligence in clinical gait analysis, which affects the identification and management of gait abnormalities found in various diseases. Machine learning (ML) and artificial neural networks (ANNs) are the most often utilized AI methods in gait analysis. By comparing the predictive capability of different AI algorithms in published studies, we evaluate their potential for gait analysis in different situations. Furthermore, the current challenges and future directions of gait analysis and AI research are discussed, which will also provide valuable reference information for investors in this field.

OptEF-BCI: An Optimization-Based Hybrid EEG and fNIRS-Brain Computer Interface

Multimodal data fusion (electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS)) has been developed as an important neuroimaging research field in order to circumvent the inherent limitations of individual modalities by combining complementary information from other modalities. This study employed an optimization-based feature selection algorithm to systematically investigate the complementary nature of multimodal fused features. After preprocessing the acquired data of both modalities (i.e., EEG and fNIRS), the temporal statistical features were computed separately with a 10 s interval for each modality. The computed features were fused to create a training vector. A wrapper-based binary enhanced whale optimization algorithm (E-WOA) was used to select the optimal/efficient fused feature subset using the support-vector-machine-based cost function. An online dataset of 29 healthy individuals was used to evaluate the performance of the proposed methodology. The findings suggest that the proposed approach enhances the classification performance by evaluating the degree of complementarity between characteristics and selecting the most efficient fused subset. The binary E-WOA feature selection approach showed a high classification rate (94.22 ± 5.39%). The classification performance exhibited a 3.85% increase compared with the conventional whale optimization algorithm. The proposed hybrid classification framework outperformed both the individual modalities and traditional feature selection classification (p < 0.01). These findings indicate the potential efficacy of the proposed framework for several neuroclinical applications.

Metaheuristic Optimization-Based Feature Selection for Imagery and Arithmetic Tasks: An fNIRS Study

In recent decades, the brain-computer interface (BCI) has emerged as a leading area of research. The feature selection is vital to reduce the dataset's dimensionality, increase the computing effectiveness, and enhance the BCI's performance. Using activity-related features leads to a high classification rate among the desired tasks. This study presents a wrapper-based metaheuristic feature selection framework for BCI applications using functional near-infrared spectroscopy (fNIRS). Here, the temporal statistical features (i.e., the mean, slope, maximum, skewness, and kurtosis) were computed from all the available channels to form a training vector. Seven metaheuristic optimization algorithms were tested for their classification performance using a k-nearest neighbor-based cost function: particle swarm optimization, cuckoo search optimization, the firefly algorithm, the bat algorithm, flower pollination optimization, whale optimization, and grey wolf optimization (GWO). The presented approach was validated based on an available online dataset of motor imagery (MI) and mental arithmetic (MA) tasks from 29 healthy subjects. The results showed that the classification accuracy was significantly improved by utilizing the features selected from the metaheuristic optimization algorithms relative to those obtained from the full set of features. All of the abovementioned metaheuristic algorithms improved the classification accuracy and reduced the feature vector size. The GWO yielded the highest average classification rates (p < 0.01) of 94.83 ± 5.5%, 92.57 ± 6.9%, and 85.66 ± 7.3% for the MA, MI, and four-class (left- and right-hand MI, MA, and baseline) tasks, respectively. The presented framework may be helpful in the training phase for selecting the appropriate features for robust fNIRS-based BCI applications.

Driving drowsiness detection using spectral signatures of EEG-based neurophysiology

Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks.

Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics.

Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen’s kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods.

Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.

A Hybrid GCN and Filter-Based Framework for Channel and Feature Selection: An fNIRS-BCI Study

In this study, a channel and feature selection methodology is devised for brain-computer interface (BCI) applications using functional near-infrared spectroscopy (fNIRS). A graph convolutional network (GCN) is employed to select the appropriate and correlated fNIRS channels. Furthermore, in the feature extraction phase, the performance of two filter-based feature selection algorithms, (i) the minimum redundancy maximum relevance (mRMR) and (ii) ReliefF, is investigated. The five most commonly used temporal statistical features (i.e., mean, slope, maximum, skewness, and kurtosis) are used, whereas the conventional support vector machine (SVM) is utilized as a classifier for training and testing. The proposed methodology is validated using an available online dataset of motor imagery (left- and right-hand), mental arithmetic, and baseline tasks. First, the efficacy of the proposed methodology is shown for two-class BCI applications (i.e., left- vs. right-hand motor imagery and mental arithmetic vs. baseline). Second, the proposed framework is applied to four-class BCI applications (i.e., left- vs. right-hand motor imagery vs. mental arithmetic vs. baseline). The results show that the number of appropriate channels and features was significantly reduced, resulting in a significant increase in classification accuracy for both two-class and four-class BCI applications, respectively. Furthermore, both mRMR (i.e., 87.8% for motor imagery, 87.1% for mental arithmetic, and 78.7% for four-class) and ReliefF (i.e., 90.7% for motor imagery, 93.7% for mental arithmetic, and 81.6% for four-class) yielded high average classification accuracy $\left(p<0.05\right)$ . However, the results of the ReliefF algorithm are more stable and significant.

Benchmarking framework for machine learning classification from fNIRS data

Background

While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods

We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion

Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.

Single-leg stance on a challenging surface can enhance cortical activation in the right hemisphere - A case study

Maintaining body balance, whether static or dynamic, is critical in performing everyday activities and developing and optimizing basic motor skills. This study investigates how a professional alpine skier's brain activates on the contralateral side during a single-leg stance. Continuous-wave functional near-infrared spectroscopy (fNIRS) signals were recorded with sixteen sources and detectors over the motor cortex to investigate brain hemodynamics. Three different tasks were performed: barefooted walk (BFW), right-leg stance (RLS), and left-leg stance (LLS). The signal processing pipeline includes channel rejection, the conversation of raw intensities into hemoglobin concentration changes using modified Beer-Lambert law, baseline zero-adjustments, z-normalization, and temporal filtration. The hemodynamic brain signal was estimated using a general linear model with a 2-gamma function. Measured activations (t-values) with p-value <0.05 were only considered as statistically significant active channels. Compared to all other conditions, BFW has the lowest brain activation. LLS is associated with more contralateral brain activation than RLS. During LLS, higher brain activation was observed across all brain regions. The right hemisphere has comparatively more activated regions-of-interest. Higher ΔHbO demands in the dorsolateral prefrontal, pre-motor, supplementary motor cortex, and primary motor cortex were observed in the right hemisphere relative to the left which explains higher energy demands for balancing during LLS. Broca's temporal lobe was also activated during both LLS and RLS. Comparing the results with BFW- which is considered the most realistic walking condition-, it is concluded that higher demands of ΔHbO predict higher motor control demands for balancing. The participant struggled with balance during the LLS, showing higher ΔHbO in both hemispheres compared to two other conditions, which indicates the higher requirement for motor control to maintain balance. A post-physiotherapy exercise program is expected to improve balance during LLS, leading to fewer changes to ΔHbO.

Impacts of simplifying articulation movements imagery to speech imagery BCI performance

Objective.Speech imagery (SI) can be used as a reliable, natural, and user-friendly activation task for the development of brain-computer interface (BCI), which empowers individuals with severe disabilities to interact with their environment. The functional near-infrared spectroscopy (fNIRS) is advanced as one of the most suitable brain imaging methods for developing BCI systems owing to its advantages of being non-invasive, portable, insensitive to motion artifacts, and having relatively high spatial resolution.Approach.To improve the classification performance of SI BCI based on fNIRS, a novel paradigm was developed in this work by simplifying the articulation movements in SI to make the articulation movement differences clearer between different words imagery tasks. A SI BCI was proposed to directly answer questions by covertly rehearsing the word '' or '' ('yes' or 'no' in English), and an unconstrained rest task also was contained in this BCI. The articulation movements of SI were simplified by retaining only the movements of the jaw and lips of vowels in Chinese Pinyin for words '' and ''.Main results.Compared with conventional speech imagery, simplifying the articulation movements in SI could generate more different brain activities among different tasks, which led to more differentiable temporal features and significantly higher classification performance. The average 3-class classification accuracies of the proposed paradigm across all 20 participants reached 69.6% and 60.2% which were about 10.8% and 5.6% significantly higher than those of the conventional SI paradigm operated in the 0-10 s and 0-2.5 s time windows, respectively.Significance.These results suggested that simplifying the articulation movements in SI is promising for improving the classification performance of intuitive BCIs based on speech imagery.

Explainable artificial intelligence model to predict brain states from fNIRS signals

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode's output onto the input variables for fNIRS signals is described here. Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model's output in terms of the model's input. Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations. Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

Applying antagonistic activation pattern to the single-trial classification of mental arithmetic

Background

At present, the application of fNIRS in the field of brain-computer interface (BCI) is being a hot topic. By fNIRS-BCI, the brain realizes the control of external devices. A state-of-the-art BCI system has five steps which are cerebral cortex signal acquisition, data pre-processing, feature selection and extraction, feature classification and application interface. Proper feature selection and extraction are crucial to the final fNIRS-BCI effect. This paper proposes a feature selection and extraction method for the mental arithmetic task. Specifically, we modified the antagonistic activation pattern approach and used the combination of antagonistic activation patterns to extract features for enhancement of the classification accuracy with low calculation costs.

Methods

Experiments are conducted on an open-acquisition dataset including fNIRS signals of eight healthy subjects of mental arithmetic (MA) tasks and rest tasks. First, the signals are filtered using band-pass filters to remove noise. Second, channels are selected by prior knowledge about antagonistic activation patterns. We used cerebral blood volume (CBV) and cerebral oxygen exchange (COE) of selected each channel to build novel attributes. Finally, we proposed three groups of attributes which are CBV, COE and CBV + COE. Based on attributes generated by the proposed method, we calculated temporal statistical measures (average, variance, maximum, minimum and slope). Any two of five statistical measures were combined as feature sets.

Main results

With the LDA, QDA, and SVM classifiers, the proposed method obtained higher classification accuracies the basic control method. The maximum classification accuracies achieved by the proposed method are 67.45 ± 14.56% with LDA classifier, 89.73 ± 5.71% with QDA classifier, and 87.04 ± 6.88% with SVM classifier. The novel method reduced the running time by 3.75 times compared with the method incorporating all channels into the feature set. Therefore, the novel method reduces the computational costs while maintaining high classification accuracy. The results are validated by another open-access dataset including MA and rest tasks of 29 healthy subjects.

Investigating effortful speech perception using fNIRS and pupillometry measures

The current study examined the neural mechanisms for mental effort and its correlation to speech perception using functional near-infrared spectroscopy (fNIRS) in listeners with normal hearing (NH). Data were collected while participants listened and responded to unprocessed and degraded sentences, where words were presented in grammatically correct or shuffled order. Effortful listening and task difficulty due to stimulus manipulations was confirmed using a subjective questionnaire and a well-established objective measure of mental effort - pupillometry. fNIRS measures focused on cortical responses in two a priori regions of interest, the left auditory cortex (AC) and lateral frontal cortex (LFC), which are closely related to auditory speech perception and listening effort, respectively. We examined the relations between the two objective measures and behavioral measures of speech perception (task performance) and task difficulty.

Results

demonstrated that changes in pupil dilation were positively correlated with the self-reported task difficulty levels and negatively correlated with the task performance scores. A significant and negative correlation between the two behavioral measures was also found. That is, as perceived task demands increased and task performance scores decreased, pupils dilated more. fNIRS measures (cerebral oxygenation) in the left AC and LFC were both negatively correlated with the self-reported task difficulty levels and positively correlated with task performance scores. These results suggest that pupillometry measures can indicate task demands and listening effort; whereas, fNIRS measures using a similar paradigm seem to reflect speech processing, but not effort.

A General and Scalable Vision Framework for Functional Near-Infrared Spectroscopy Classification

Functional near-infrared spectroscopy (fNIRS), a non-invasive optical technique, is widely used to monitor brain activities for disease diagnosis and brain-computer interfaces (BCIs). Deep learning-based fNIRS classification faces three major barriers: limited datasets, confusing evaluation criteria, and domain barriers. We apply more appropriate evaluation methods to three open-access datasets to solve the first two barriers. For domain barriers, we propose a general and scalable vision fNIRS framework that converts multi-channel fNIRS signals into multi-channel virtual images using the Gramian angular difference field (GADF). We use the framework to train state-of-the-art visual models from computer vision (CV) within a few minutes, and the classification performance is competitive with the latest fNIRS models. In cross-validation experiments, visual models achieve the highest average classification results of 78.68% and 73.92% on mental arithmetic and word generation tasks, respectively. Although visual models are slightly lower than the fNIRS models on unilateral finger- and foot-tapping tasks, the F1-score and kappa coefficient indicate that these differences are insignificant in subject-independent experiments. Furthermore, we study fNIRS signal representations and the classification performance of sequence-to-image methods. We hope to introduce rich achievements from the CV domain to improve fNIRS classification research.

Effects of degraded speech processing and binaural unmasking investigated using functional near-infrared spectroscopy (fNIRS)

The present study aimed to investigate the effects of degraded speech perception and binaural unmasking using functional near-infrared spectroscopy (fNIRS). Normal hearing listeners were tested when attending to unprocessed or vocoded speech, presented to the left ear at two speech-to-noise ratios (SNRs). Additionally, by comparing monaural versus diotic masker noise, we measured binaural unmasking. Our primary research question was whether the prefrontal cortex and temporal cortex responded differently to varying listening configurations. Our a priori regions of interest (ROIs) were located at the left dorsolateral prefrontal cortex (DLPFC) and auditory cortex (AC). The left DLPFC has been reported to be involved in attentional processes when listening to degraded speech and in spatial hearing processing, while the AC has been reported to be sensitive to speech intelligibility. Comparisons of cortical activity between these two ROIs revealed significantly different fNIRS response patterns. Further, we showed a significant and positive correlation between self-reported task difficulty levels and fNIRS responses in the DLPFC, with a negative but non-significant correlation for the left AC, suggesting that the two ROIs played different roles in effortful speech perception. Our secondary question was whether activity within three sub-regions of the lateral PFC (LPFC) including the DLPFC was differentially affected by varying speech-noise configurations. We found significant effects of spectral degradation and SNR, and significant differences in fNIRS response amplitudes between the three regions, but no significant interaction between ROI and speech type, or between ROI and SNR. When attending to speech with monaural and diotic noises, participants reported the latter conditions being easier; however, no significant main effect of masker condition on cortical activity was observed. For cortical responses in the LPFC, a significant interaction between SNR and masker condition was observed. These findings suggest that binaural unmasking affects cortical activity through improving speech reception threshold in noise, rather than by reducing effort exerted.

LASSO Homotopy-Based Sparse Representation Classification for fNIRS-BCI

Brain-computer interface (BCI) systems based on functional near-infrared spectroscopy (fNIRS) have been used as a way of facilitating communication between the brain and peripheral devices. The BCI provides an option to improve the walking pattern of people with poor walking dysfunction, by applying a rehabilitation process. A state-of-the-art step-wise BCI system includes data acquisition, pre-processing, channel selection, feature extraction, and classification. In fNIRS-based BCI (fNIRS-BCI), channel selection plays a vital role in enhancing the classification accuracy of the BCI problem. In this study, the concentration of blood oxygenation (HbO) in a resting state and in a walking state was used to decode the walking activity and the resting state of the subject, using channel selection by Least Absolute Shrinkage and Selection Operator (LASSO) homotopy-based sparse representation classification. The fNIRS signals of nine subjects were collected from the left hemisphere of the primary motor cortex. The subjects performed the task of walking on a treadmill for 10 s, followed by a 20 s rest. Appropriate filters were applied to the collected signals to remove motion artifacts and physiological noises. LASSO homotopy-based sparse representation was used to select the most significant channels, and then classification was performed to identify walking and resting states. For comparison, the statistical spatial features of mean, peak, variance, and skewness, and their combination, were used for classification. The classification results after channel selection were then compared with the classification based on the extracted features. The classifiers used for both methods were linear discrimination analysis (LDA), support vector machine (SVM), and logistic regression (LR). The study found that LASSO homotopy-based sparse representation classification successfully discriminated between the walking and resting states, with a better average classification accuracy (p < 0.016) of 91.32%. This research provides a step forward in improving the classification accuracy of fNIRS-BCI systems. The proposed methodology may also be used for rehabilitation purposes, such as controlling wheelchairs and prostheses, as well as an active rehabilitation training technique for patients with motor dysfunction.

Analyzing Classification Performance of fNIRS-BCI for Gait Rehabilitation Using Deep Neural Networks

This research presents a brain-computer interface (BCI) framework for brain signal classification using deep learning (DL) and machine learning (ML) approaches on functional near-infrared spectroscopy (fNIRS) signals. fNIRS signals of motor execution for walking and rest tasks are acquired from the primary motor cortex in the brain's left hemisphere for nine subjects. DL algorithms, including convolutional neural networks (CNNs), long short-term memory (LSTM), and bidirectional LSTM (Bi-LSTM) are used to achieve average classification accuracies of 88.50%, 84.24%, and 85.13%, respectively. For comparison purposes, three conventional ML algorithms, support vector machine (SVM), k-nearest neighbor (k-NN), and linear discriminant analysis (LDA) are also used for classification, resulting in average classification accuracies of 73.91%, 74.24%, and 65.85%, respectively. This study successfully demonstrates that the enhanced performance of fNIRS-BCI can be achieved in terms of classification accuracy using DL approaches compared to conventional ML approaches. Furthermore, the control commands generated by these classifiers can be used to initiate and stop the gait cycle of the lower limb exoskeleton for gait rehabilitation.

Characterization of Bimanual Cyclical Tasks from Single-trial EEG-fNIRS Measurements

Robot-assisted bimanual training is promising to improve motor function and cortical reorganization for hemiparetic stroke patients. Closing the rehabilitation training loop with neurofeedback can help refine training protocols in time for better engagements and outcomes. However, due to the low signal-to- noise ratio (SNR) and non-stationary properties of neural signals, reliable characterization of bimanual training-induced neural activities from single-trial measurement is challenging. In this study, ten human participants were recruited conducting robot-assisted bimanual cyclical tasks (in-phase, 90° out-of-phase, and anti-phase) when concurrent electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded. A unified EEG-fNIRS bimodal signal processing framework was proposed to characterize neural activities induced by three types of bimanual cyclical tasks. In this framework, novel artifact removal methods were used to improve the SNR and the task-related component analysis (TRCA) was introduced to increase the reproducibility of EEG-fNIRS bimodal features. The optimized features were transformed into low-dimensional indicators to reliably characterize bimanual training-induced neural activation. The SVM classification results of three bimanual cyclical tasks revealed a good discrimination ability of EEG-fNIRS bimodal indicators (90.1%), which was higher than that using EEG (74.8%) or fNIRS (82.2%) alone, supporting the proposed method as a feasible technique to characterize neural activities during robot-assisted bimanual training.

Acupuncture enhances brain function in patients with mild cognitive impairment: evidence from a functional-near infrared spectroscopy study

Mild cognitive impairment (MCI) is a precursor to Alzheimer’s disease. It is imperative to develop a proper treatment for this neurological disease in the aging society. This observational study investigated the effects of acupuncture therapy on MCI patients. Eleven healthy individuals and eleven MCI patients were recruited for this study. Oxy- and deoxy-hemoglobin signals in the prefrontal cortex during working-memory tasks were monitored using functional near-infrared spectroscopy. Before acupuncture treatment, working-memory experiments were conducted for healthy control (HC) and MCI groups (MCI-0), followed by 24 sessions of acupuncture for the MCI group. The acupuncture sessions were initially carried out for 6 weeks (two sessions per week), after which experiments were performed again on the MCI group (MCI-1). This was followed by another set of acupuncture sessions that also lasted for 6 weeks, after which the experiments were repeated on the MCI group (MCI-2). Statistical analyses of the signals and classifications based on activation maps as well as temporal features were performed. The highest classification accuracies obtained using binary connectivity maps were 85.7% HC vs. MCI-0, 69.5% HC vs. MCI-1, and 61.69% HC vs. MCI-2. The classification accuracies using the temporal features mean from 5 seconds to 28 seconds and maximum (i.e, max(5:28 seconds)) values were 60.6% HC vs. MCI-0, 56.9% HC vs. MCI-1, and 56.4% HC vs. MCI-2. The results reveal that there was a change in the temporal characteristics of the hemodynamic response of MCI patients due to acupuncture. This was reflected by a reduction in the classification accuracy after the therapy, indicating that the patients’ brain responses improved and became comparable to those of healthy subjects. A similar trend was reflected in the classification using the image feature. These results indicate that acupuncture can be used for the treatment of MCI patients.

Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e., optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky–Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.

Most favorable stimulation duration in the sensorimotor cortex for fNIRS-based BCI

One of the primary objectives of the brain-computer interface (BCI) is to obtain a command with higher classification accuracy within the shortest possible time duration. Therefore, this study evaluates several stimulation durations to propose a duration that can yield the highest classification accuracy. Furthermore, this study aims to address the inherent delay in the hemodynamic responses (HRs) for the command generation time. To this end, HRs in the sensorimotor cortex were evaluated for the functional near-infrared spectroscopy (fNIRS)-based BCI. To evoke brain activity, right-hand-index finger poking and tapping tasks were used. In this study, six different stimulation durations (i.e., 1, 3, 5, 7, 10, and 15 s) were tested on 10 healthy male subjects. Upon stimulation, different temporal features and multiple time windows were utilized to extract temporal features. The extracted features were then classified using linear discriminant analysis. The classification results using the main HR showed that a 5 s stimulation duration could yield the highest classification accuracy, i.e., 74%, with a combination of the mean and maximum value features. However, the results were not significantly different from the classification accuracy obtained using the 15 s stimulation. To further validate the results, a classification using the initial dip was performed. The results obtained endorsed the finding with an average classification accuracy of 73.5% using the features of minimum peak and skewness in the 5 s window. The results based on classification using the initial dip for 5 s were significantly different from all other tested stimulation durations (p < 0.05) for all feature combinations. Moreover, from the visual inspection of the HRs, it is observed that the initial dip occurred as soon as the task started, but the main HR had a delay of more than 2 s. Another interesting finding is that impulsive stimulation in the sensorimotor cortex can result in the generation of a clearer initial dip phenomenon. The results reveal that the command for the fNIRS-based BCI can be generated using the 5 s stimulation duration. In conclusion, the use of the initial dip can reduce the time taken for the generation of commands and can be used to achieve a higher classification accuracy for the fNIRS-BCI within a 5 s task duration rather than relying on longer durations.

fNIRS Evidence for Distinguishing Patients With Major Depression and Healthy Controls

In recent years, major depressive disorder (MDD) has been shown to negatively impact physical recovery in a variety of patients. Functional near-infrared spectroscopy (fNIRS) is a tool that can potentially supplement clinical interviews and mental state examinations to establish a psychiatric diagnosis and monitor treatment progress. Thirty-two subjects, including 16 patients clinically diagnosed with MDD and 16 healthy controls (HCs), participated in the study. Brain oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) responses were recorded using a 22-channel continuous-wave fNIRS device while the subjects performed the emotional sound test. This study evaluated the difference between MDD patients and HCs using a variety of methods. In a comparison of the Pearson correlation coefficients between the HbO/HbR responses of each fNIRS channel and four scores, MDD patients and HCs had significantly different Athens Insomnia Scale (AIS) scores. By quantitative evaluation of the functional association, we found that MDD patients had aberrant functional connectivity compared with HCs. Furthermore, we concluded that compared with HCs, there were marked abnormalities in blood oxygen in the bilateral ventrolateral prefrontal cortex (VLPFC) and bilateral dorsolateral prefrontal cortex (DLPFC). Four statistical-based features extracted from HbO signals and four vector-based features from both HbO and HbR served as inputs to four simple neural networks (multilayer neural network (MNN), feedforward neural network (FNN), cascade forward neural network (CFNN) and recurrent neural network (RNN)). Through an analysis of combinations of different features, the combination of 4 common features (mean, STD, area under the receiver operating characteristic curve (AUC) and slope) yielded the highest classification accuracy of 89.74% for fear emotion. The combination of four novel feature (CBV, COE, |L | and K) resulted in a classification accuracy of 99.94% for fear emotion. The top 10 common and novel features were selected by the ReliefF feature selection algorithm, resulting in classification accuracies of 83.52% and 91.99%, respectively. This study identified the AUC and angle K as specific neuromarkers for predicting MDD across specific depression-related regions of the prefrontal cortex (PFC). These findings suggest that the fNIRS measurement of the PFC may serve as a supplementary test in routine clinical practice to further support a diagnosis of MDD.

Data Processing in Functional Near-Infrared Spectroscopy (fNIRS) Motor Control Research

FNIRS pre-processing and processing methodologies are very important-how a researcher chooses to process their data can change the outcome of an experiment. The purpose of this review is to provide a guide on fNIRS pre-processing and processing techniques pertinent to the field of human motor control research. One hundred and twenty-three articles were selected from the motor control field and were examined on the basis of their fNIRS pre-processing and processing methodologies. Information was gathered about the most frequently used techniques in the field, which included frequency cutoff filters, wavelet filters, smoothing filters, and the general linear model (GLM). We discuss the methodologies of and considerations for these frequently used techniques, as well as those for some alternative techniques. Additionally, general considerations for processing are discussed.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Comparing verbal working memory load in auditory and visual modalities using functional near-infrared spectroscopy

The verbal identity n-back task is commonly used to assess verbal working memory (VWM) capacity. Only three studies have compared brain activation during the n-back when using auditory and visual stimuli. The earliest study, a positron emission tomography study of the 3-back, found no differences in VWM-related brain activation between n-back modalities. In contrast, two subsequent functional magnetic resonance imaging (fMRI) studies of the 2-back found that auditory VWM was associated with greater left dorsolateral prefrontal cortex (DL-PFC) activation than visual VWM, perhaps suggesting that auditory VWM requires more cognitive effort than its visual counterpart. The current study aimed to assess whether DL-PFC activation (i.e., cognitive effort) differs by VWM modality. To do this, 16 younger adults completed an auditory and visual n-back, both at four levels of VWM load. Concurrently, activation of the PFC was measured using functional near-infrared spectroscopy (fNIRS), a silent neuroimaging method. We found that DL-PFC activation increased with VWM load, but it was not affected by VWM modality or the interaction between load and modality. This supports the view that both VWM modalities require similar cognitive effort, and perhaps that previous fMRI results were an artefact of scanner noise. We also found that, across conditions, DL-PFC activation was positively correlated with reaction time. This may further support DL-PFC activation as an index of cognitive effort, and fNIRS as a method to measure it.

Summary of over Fifty Years with Brain-Computer Interfaces-A Review

Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.

Enhancing Classification Performance of fNIRS-BCI by Identifying Cortically Active Channels Using the z-Score Method

A state-of-the-art brain–computer interface (BCI) system includes brain signal acquisition, noise removal, channel selection, feature extraction, classification, and an application interface. In functional near-infrared spectroscopy-based BCI (fNIRS-BCI) channel selection may enhance classification performance by identifying suitable brain regions that contain brain activity. In this study, the z-score method for channel selection is proposed to improve fNIRS-BCI performance. The proposed method uses cross-correlation to match the similarity between desired and recorded brain activity signals, followed by forming a vector of each channel’s correlation coefficients’ maximum values. After that, the z-score is calculated for each value of that vector. A channel is selected based on a positive z-score value. The proposed method is applied to an open-access dataset containing mental arithmetic (MA) and motor imagery (MI) tasks for twenty-nine subjects. The proposed method is compared with the conventional t-value method and with no channel selected, i.e., using all channels. The z-score method yielded significantly improved (p < 0.0167) classification accuracies of 87.2 ± 7.0%, 88.4 ± 6.2%, and 88.1 ± 6.9% for left motor imagery (LMI) vs. rest, right motor imagery (RMI) vs. rest, and mental arithmetic (MA) vs. rest, respectively. The proposed method is also validated on an open-access database of 17 subjects, containing right-hand finger tapping (RFT), left-hand finger tapping (LFT), and dominant side foot tapping (FT) tasks.The study shows an enhanced performance of the z-score method over the t-value method as an advancement in efforts to improve state-of-the-art fNIRS-BCI systems’ performance.