Most favorable stimulation duration in the sensorimotor cortex for fNIRS-based BCI

Hemodynamic activity is not parsimoniously tuned to index-of-difficulty in movement with dual requirements on speed-accuracy

Background

Reaching movements are crucial for daily living and rehabilitation, for which Fitts' Law describes a speed-accuracy trade-off that movement time increases with task difficulty. This study aims to investigate whether cortical activation in motor-related areas is directly linked to task difficulty as defined by Fitts' Law. Understanding this relationship provides a physiological basis for parameter selection in therapeutic exercises.

Methods

Sixteen healthy subjects performed 2D reaching movements using a rehabilitation robot, with their cortical responses detected using functional near-infrared spectroscopy (fNIRS). Task difficulty was manipulated by varying target size and distance, resulting in 3 levels of index-of-difficulty (ID). Kinematic signals were recorded alongside cortical activity to assess the relationship among movement time, task difficulty, and cortical activation.

Results

Our results showed that movement time increased with ID by 0.2974s/bit across all subjects (conditional r2 = 0.6434, p < 0.0001), and all subjects showed individual trends conforming Fitts' Law (all p < 0.001). Neither activation in BA4 nor in BA6 showed a significant correlation with ID (p > 0.05), while both the target size and distance, as well as the interaction between them, showed a significant relationship with BA4 or BA6 activation (all p < 0.05).

Conclusion

This study found that although kinematic measures supported Fitts' Law, cortical activity in motor-related areas during reaching movements did not correlate directly with task difficulty as defined by Fitts' Law. Additional factors such as muscle activation may call for different cortical control even when difficulty was identical.

CGAN-rIRN: a data-augmented deep learning approach to accurate classification of mental tasks for a fNIRS-based brain-computer interface

Functional near-infrared spectroscopy (fNIRS) is increasingly used to investigate different mental tasks for brain-computer interface (BCI) control due to its excellent environmental and motion robustness. Feature extraction and classification strategy for fNIRS signal are essential to enhance the classification accuracy of voluntarily controlled BCI systems. The limitation of traditional machine learning classifiers (MLCs) lies in manual feature engineering, which is considered as one of the drawbacks that reduce accuracy. Since the fNIRS signal is a typical multivariate time series with multi-dimensionality and complexity, it makes the deep learning classifier (DLC) ideal for classifying neural activation patterns. However, the inherent bottleneck of DLCs is the requirement of substantial-scale, high-quality labeled training data and expensive computational resources to train deep networks. The existing DLCs for classifying mental tasks do not fully consider the temporal and spatial properties of fNIRS signals. Therefore, a specifically-designed DLC is desired to classify multi-tasks with high accuracy in fNIRS-BCI. To this end, we herein propose a novel data-augmented DLC to accurately classify mental tasks, which employs a convolution-based conditional generative adversarial network (CGAN) for data augmentation and a revised Inception-ResNet (rIRN) based DLC. The CGAN is utilized to generate class-specific synthetic fNIRS signals to augment the training dataset. The network architecture of rIRN is elaborately designed in accordance with the characteristics of the fNIRS signal, with serial multiple spatial and temporal feature extraction modules (FEMs), where each FEM performs deep and multi-scale feature extraction and fusion. The results of the paradigm experiments show that the proposed CGAN-rIRN approach improves the single-trial accuracy for mental arithmetic and mental singing tasks in both the data augmentation and classifier, as compared to the traditional MLCs and the commonly used DLCs. The proposed fully data-driven hybrid deep learning approach paves a promising way to improve the classification performance of volitional control fNIRS-BCI.

Benchmarking framework for machine learning classification from fNIRS data

Background

While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods

We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion

Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.

A Potential Prognosis Indicator Based on P300 Brain-Computer Interface for Patients with Disorder of Consciousness

For patients with disorders of consciousness, such as unresponsive wakefulness syndrome (UWS) patients and minimally conscious state (MCS) patients, their long treatment cycle and high cost commonly put a heavy burden on the patient's family and society. Therefore, it is vital to accurately diagnose and predict consciousness recovery for such patients. In this paper, we explored the role of the P300 signal based on an audiovisual BCI in the classification and prognosis prediction of patients with disorders of consciousness. This experiment included 18 patients: 10 UWS patients and 8 MCS- patients. At the three-month follow-up, we defined patients with an improved prognosis (from UWS to MCS-, from UWS to MCS+, or from MCS- to MCS+) as "improved patients" and those who stayed in UWS/MCS as "not improved patients". First, we compared and analyzed different types of patients, and the results showed that the P300 detection accuracy rate of "improved" patients was significantly higher than that of "not improved" patients. Furthermore, the P300 detection accuracy of traumatic brain injury (TBI) patients was significantly higher than that of non-traumatic brain injury (NTBI, including acquired brain injury and cerebrovascular disease) patients. We also found that there was a positive linear correlation between P300 detection accuracy and CRS-R score, and patients with higher P300 detection accuracy were likely to achieve higher CRS-R scores. In addition, we found that the patients with higher P300 detection accuracies tend to have better prognosis in this audiovisual BCI. These findings indicate that the detection accuracy of P300 is significantly correlated with the level of consciousness, etiology, and prognosis of patients. P300 can be used to represent the preservation level of consciousness in clinical neurophysiology and predict the possibility of recovery in patients with disorders of consciousness.

Applying antagonistic activation pattern to the single-trial classification of mental arithmetic

Background

At present, the application of fNIRS in the field of brain-computer interface (BCI) is being a hot topic. By fNIRS-BCI, the brain realizes the control of external devices. A state-of-the-art BCI system has five steps which are cerebral cortex signal acquisition, data pre-processing, feature selection and extraction, feature classification and application interface. Proper feature selection and extraction are crucial to the final fNIRS-BCI effect. This paper proposes a feature selection and extraction method for the mental arithmetic task. Specifically, we modified the antagonistic activation pattern approach and used the combination of antagonistic activation patterns to extract features for enhancement of the classification accuracy with low calculation costs.

Methods

Experiments are conducted on an open-acquisition dataset including fNIRS signals of eight healthy subjects of mental arithmetic (MA) tasks and rest tasks. First, the signals are filtered using band-pass filters to remove noise. Second, channels are selected by prior knowledge about antagonistic activation patterns. We used cerebral blood volume (CBV) and cerebral oxygen exchange (COE) of selected each channel to build novel attributes. Finally, we proposed three groups of attributes which are CBV, COE and CBV + COE. Based on attributes generated by the proposed method, we calculated temporal statistical measures (average, variance, maximum, minimum and slope). Any two of five statistical measures were combined as feature sets.

Main results

With the LDA, QDA, and SVM classifiers, the proposed method obtained higher classification accuracies the basic control method. The maximum classification accuracies achieved by the proposed method are 67.45 ± 14.56% with LDA classifier, 89.73 ± 5.71% with QDA classifier, and 87.04 ± 6.88% with SVM classifier. The novel method reduced the running time by 3.75 times compared with the method incorporating all channels into the feature set. Therefore, the novel method reduces the computational costs while maintaining high classification accuracy. The results are validated by another open-access dataset including MA and rest tasks of 29 healthy subjects.

A pediatric near-infrared spectroscopy brain-computer interface based on the detection of emotional valence

Brain-computer interfaces (BCIs) are being investigated as an access pathway to communication for individuals with physical disabilities, as the technology obviates the need for voluntary motor control. However, to date, minimal research has investigated the use of BCIs for children. Traditional BCI communication paradigms may be suboptimal given that children with physical disabilities may face delays in cognitive development and acquisition of literacy skills. Instead, in this study we explored emotional state as an alternative access pathway to communication. We developed a pediatric BCI to identify positive and negative emotional states from changes in hemodynamic activity of the prefrontal cortex (PFC). To train and test the BCI, 10 neurotypical children aged 8–14 underwent a series of emotion-induction trials over four experimental sessions (one offline, three online) while their brain activity was measured with functional near-infrared spectroscopy (fNIRS). Visual neurofeedback was used to assist participants in regulating their emotional states and modulating their hemodynamic activity in response to the affective stimuli. Child-specific linear discriminant classifiers were trained on cumulatively available data from previous sessions and adaptively updated throughout each session. Average online valence classification exceeded chance across participants by the last two online sessions (with 7 and 8 of the 10 participants performing better than chance, respectively, in Sessions 3 and 4). There was a small significant positive correlation with online BCI performance and age, suggesting older participants were more successful at regulating their emotional state and/or brain activity. Variability was seen across participants in regards to BCI performance, hemodynamic response, and discriminatory features and channels. Retrospective offline analyses yielded accuracies comparable to those reported in adult affective BCI studies using fNIRS. Affective fNIRS-BCIs appear to be feasible for school-aged children, but to further gauge the practical potential of this type of BCI, replication with more training sessions, larger sample sizes, and end-users with disabilities is necessary.

Non-invasive transcranial electrical brain stimulation guided by functional near-infrared spectroscopy for targeted neuromodulation: A review

One of the primary goals in cognitive neuroscience is to understand the neural mechanisms on which cognition is based. Researchers are trying to find how cognitive mechanisms are related to oscillations generated due to brain activity. The research focused on this topic has been considerably aided by developing non-invasive brain stimulation techniques. The dynamics of brain networks and the resultant behavior can be affected by non-invasive brain stimulation techniques, which make their use a focus of interest in many experiments and clinical fields. One essential non-invasive brain stimulation technique is transcranial electrical stimulation (tES), subdivided into transcranial direct and alternating current stimulation. tES has recently become more well-known because of the effective results achieved in treating chronic conditions. In addition, there has been exceptional progress in the interpretation and feasibility of tES techniques. Summarizing the beneficial effects of tES, this article provides an updated depiction of what has been accomplished to date, brief history, and the open questions that need to be addressed in the future. An essential issue in the field of tES is stimulation duration. This review briefly covers the stimulation durations that have been utilized in the field while monitoring the brain using functional-near infrared spectroscopy-based brain imaging.

Acupuncture enhances brain function in patients with mild cognitive impairment: evidence from a functional-near infrared spectroscopy study

Mild cognitive impairment (MCI) is a precursor to Alzheimer’s disease. It is imperative to develop a proper treatment for this neurological disease in the aging society. This observational study investigated the effects of acupuncture therapy on MCI patients. Eleven healthy individuals and eleven MCI patients were recruited for this study. Oxy- and deoxy-hemoglobin signals in the prefrontal cortex during working-memory tasks were monitored using functional near-infrared spectroscopy. Before acupuncture treatment, working-memory experiments were conducted for healthy control (HC) and MCI groups (MCI-0), followed by 24 sessions of acupuncture for the MCI group. The acupuncture sessions were initially carried out for 6 weeks (two sessions per week), after which experiments were performed again on the MCI group (MCI-1). This was followed by another set of acupuncture sessions that also lasted for 6 weeks, after which the experiments were repeated on the MCI group (MCI-2). Statistical analyses of the signals and classifications based on activation maps as well as temporal features were performed. The highest classification accuracies obtained using binary connectivity maps were 85.7% HC vs. MCI-0, 69.5% HC vs. MCI-1, and 61.69% HC vs. MCI-2. The classification accuracies using the temporal features mean from 5 seconds to 28 seconds and maximum (i.e, max(5:28 seconds)) values were 60.6% HC vs. MCI-0, 56.9% HC vs. MCI-1, and 56.4% HC vs. MCI-2. The results reveal that there was a change in the temporal characteristics of the hemodynamic response of MCI patients due to acupuncture. This was reflected by a reduction in the classification accuracy after the therapy, indicating that the patients’ brain responses improved and became comparable to those of healthy subjects. A similar trend was reflected in the classification using the image feature. These results indicate that acupuncture can be used for the treatment of MCI patients.

Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e., optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky–Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.