Open-Access fNIRS Dataset for Classification of Unilateral Finger- and Foot-Tapping

Opening the dialogue: A preliminary exploration of hair color, hair cleanliness, light, and motion effects on fNIRS signal quality

INTRODUCTION

Functional near-infrared spectroscopy (fNIRS) is a promising tool for studying brain activity, offering advantages such as portability and affordability. However, challenges in data collection persist due to factors like participant physiology, environmental light, and gross-motor movements, with limited literature on their impact on fNIRS signal quality. This study addresses four potentially influential factors-hair color, hair cleanliness, environmental light, and gross-motor movements-on fNIRS signal quality. Our aim is to raise awareness and offer insights for future fNIRS research.

METHODS

Six participants (4 Females, 2 Males) took part in four different experiments investigating the effects of hair color, hair cleanliness, environmental light, and gross-motor movements on fNIRS signal quality. Participants in Experiment 1, categorized by hair color, completed a finger-tapping task in a between-subjects block design. Signal quality was compared between each hair color. Participants in Experiments 2 and 3 completed a finger-tapping task in a within-subjects block design, with signal quality being compared across hair cleanliness (i.e., five consecutive days without washing the hair) and environmental light (i.e., sunlight, artificial light, no light, etc.), respectively. Experiment 4 assessed three gross-motor movements (i.e., walking, turning and nodding the head) in a within-subjects block design. Motor movements were then compared to resting blocks. Signal quality was evaluated using Scalp Coupling Index (SCI) measurements.

RESULTS

Lighter hair produced better signals than dark hair, while the impact of environmental light remains uncertain. Hair cleanliness showed no significant effects, but gross motor movements notably reduced signal quality.

CONCLUSION

Our results suggest that hair color, environmental light, and gross-motor movements affect fNIRS signal quality while hair cleanliness does not. Nevertheless, future studies with larger sample sizes are warranted to fully understand these effects. To advance future research, comprehensive documentation of participant demographics and lab conditions, along with signal quality analyses, is essential.

Enhancing Classification Accuracy with Integrated Contextual Gate Network: Deep Learning Approach for Functional Near-Infrared Spectroscopy Brain-Computer Interface Application

Brain-computer interface (BCI) systems include signal acquisition, preprocessing, feature extraction, classification, and an application phase. In fNIRS-BCI systems, deep learning (DL) algorithms play a crucial role in enhancing accuracy. Unlike traditional machine learning (ML) classifiers, DL algorithms eliminate the need for manual feature extraction. DL neural networks automatically extract hidden patterns/features within a dataset to classify the data. In this study, a hand-gripping (closing and opening) two-class motor activity dataset from twenty healthy participants is acquired, and an integrated contextual gate network (ICGN) algorithm (proposed) is applied to that dataset to enhance the classification accuracy. The proposed algorithm extracts the features from the filtered data and generates the patterns based on the information from the previous cells within the network. Accordingly, classification is performed based on the similar generated patterns within the dataset. The accuracy of the proposed algorithm is compared with the long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). The proposed ICGN algorithm yielded a classification accuracy of 91.23 ± 1.60%, which is significantly (p < 0.025) higher than the 84.89 ± 3.91 and 88.82 ± 1.96 achieved by LSTM and Bi-LSTM, respectively. An open access, three-class (right- and left-hand finger tapping and dominant foot tapping) dataset of 30 subjects is used to validate the proposed algorithm. The results show that ICGN can be efficiently used for the classification of two- and three-class problems in fNIRS-based BCI applications.

On decoding of rapid motor imagery in a diverse population using a high-density NIRS device

Introduction

Functional near-infrared spectroscopy (fNIRS) aims to infer cognitive states such as the type of movement imagined by a study participant in a given trial using an optical method that can differentiate between oxygenation states of blood in the brain and thereby indirectly between neuronal activity levels. We present findings from an fNIRS study that aimed to test the applicability of a high-density (>3000 channels) NIRS device for use in short-duration (2 s) left/right hand motor imagery decoding in a diverse, but not explicitly balanced, subject population. A side aim was to assess relationships between data quality, self-reported demographic characteristics, and brain-computer interface (BCI) performance, with no subjects rejected from recruitment or analysis.

Methods

BCI performance was quantified using several published methods, including subject-specific and subject-independent approaches, along with a high-density fNIRS decoder previously validated in a separate study.

Results

We found that decoding of motor imagery on this population proved extremely challenging across all tested methods. Overall accuracy of the best-performing method (the high-density decoder) was 59.1 +/- 6.7% after excluding subjects where almost no optode-scalp contact was made over motor cortex and 54.7 +/- 7.6% when all recorded sessions were included. Deeper investigation revealed that signal quality, hemodynamic responses, and BCI performance were all strongly impacted by the hair phenotypical and demographic factors under investigation, with over half of variance in signal quality explained by demographic factors alone.

Discussion

Our results contribute to the literature reporting on challenges in using current-generation NIRS devices on subjects with long, dense, dark, and less pliable hair types along with the resulting potential for bias. Our findings confirm the need for increased focus on these populations, accurate reporting of data rejection choices across subject intake, curation, and final analysis in general, and signal a need for NIRS optode designs better optimized for the general population to facilitate more robust and inclusive research outcomes.

Feasibility of local interpretable model-agnostic explanations (LIME) algorithm as an effective and interpretable feature selection method: comparative fNIRS study

Many feature selection methods have been evaluated in functional near-infrared spectroscopy (fNIRS)-related studies. The local interpretable model-agnostic explanation (LIME) algorithm is a feature selection method for fNIRS datasets that has not yet been validated; the demand for its validation is increasing. To this end, we assessed the feature selection performance of LIME for fNIRS datasets in terms of classification accuracy. A comparative analysis was conducted for the benchmark (classification accuracy obtained without applying any feature selection method), LIME, two filter-based methods (minimum-redundancy maximum-relevance and t-test), and one wrapper-based method (sequential forward selection). To ensure the fairness and reliability of the performance evaluation, several open-access fNIRS datasets were used. The analysis revealed that LIME greatly outperformed the other feature selection methods in most cases and could achieve a statistically significantly better classification accuracy than that of the benchmark methods. These findings implied the effectiveness of LIME as a feature selection approach for fNIRS datasets.

Modulating Cortical Hemodynamic Activity in Parkinson's Disease Using Focal Transcranial Direct Current Stimulation: A Pilot Functional Near-infrared Spectroscopy Study

Reduced thalamocortical facilitation of the motor cortex in PD leads to characteristic motor deficits such as bradykinesia. Recent research has highlighted improved motor function following tDCS, but a lack of neurophysiological evidence limits the progress of tDCS as an adjunctive therapy. Here, we tested the hypothesis that tDCS may modulate M1 hemodynamic activity in PD and healthy using functional near-infrared spectroscopy (fNIRS). In this randomized crossover experiment, fourteen PD and twelve healthy control participants attended three laboratory sessions and performed a regulated (3 Hz) right index finger tapping task before and after receiving tDCS. On each visit, participants received either anodal, cathodal, or sham tDCS applied over M1. Hemodynamic activity of M1 was quantified using fNIRS. Significant task related activity was observed in M1 and the inferior parietal lobe in PD and healthy (p < 0.05). PD additionally recruited the dorsal premotor cortex. During tDCS, while at rest, anodal and cathodal tDCS significantly increased the oxygenated hemoglobin concentration of M1 compared to sham (t62 = 4.09 and t62 = 4.25, respectively). Task related hemodynamic activity was unchanged following any tDCS intervention (p > 0.05). Task related hemodynamic activity of M1 is not modulated by tDCS in PD or healthy. During tDCS, both anodal and cathodal stimulation cause a significant increase of M1 oxygenation, the clinical significance of which remains to be clarified.

Exploring fNIRS-Based Brain State Recognition and Visualization through the use of Explainable Convolutional Neural Networks

Functional near infrared spectroscopy (fNIRS) is a neuroimaging technique that has grown vigorously in recent years. With noticeable attention, machine learning methods have also been applied to fNIRS. However, the current approach lacks interpretability of the results. In recent years, the utilization and investigation of fNIRS have experienced significant growth and are now being utilized in clinical research. However, the collection of clinical fNIRS data is limited in sample size. Therefore, our aim is to utilize the collected fNIRS data from all channels and achieve interpretable analysis results with minimal human manipulation, channel selection or feature extraction. We developed an fNIRS-based interpretable model and used class-specific gradient information to visualize the biomarkers captured by the model via locating the important region. The accuracy of our model's classification was 6% higher than that of the conventional SVM method under within-subject classification. The model focuses on signals from the left brain in the classification of right-hand finger tapping task, while in the task of classifying left-handed movements, the model relies on signals from the right brain. These results were consistent with current understanding of physiology.Clinical Relevance- The machine learning-based fNIRS model has the potential to be used for the diagnosis and prediction of therapeutic efficacy in clinical settings.

Multimodal motor imagery decoding method based on temporal spatial feature alignment and fusion

Objective. A motor imagery-based brain-computer interface (MI-BCI) translates spontaneous movement intention from the brain to outside devices. Multimodal MI-BCI that uses multiple neural signals contains rich common and complementary information and is promising for enhancing the decoding accuracy of MI-BCI. However, the heterogeneity of different modalities makes the multimodal decoding task difficult. How to effectively utilize multimodal information remains to be further studied.Approach. In this study, a multimodal MI decoding neural network was proposed. Spatial feature alignment losses were designed to enhance the feature representations extracted from the heterogeneous data and guide the fusion of features from different modalities. An attention-based modality fusion module was built to align and fuse the features in the temporal dimension. To evaluate the proposed decoding method, a five-class MI electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) dataset were constructed.Main results and significance. The comparison experimental results showed that the proposed decoding method achieved higher decoding accuracy than the compared methods on both the self-collected dataset and a public dataset. The ablation results verified the effectiveness of each part of the proposed method. Feature distribution visualization results showed that the proposed losses enhance the feature representation of EEG and fNIRS modalities. The proposed method based on EEG and fNIRS modalities has significant potential for improving decoding performance of MI tasks.

Benchmarking framework for machine learning classification from fNIRS data

Background

While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods

We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion

Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.

A General and Scalable Vision Framework for Functional Near-Infrared Spectroscopy Classification

Functional near-infrared spectroscopy (fNIRS), a non-invasive optical technique, is widely used to monitor brain activities for disease diagnosis and brain-computer interfaces (BCIs). Deep learning-based fNIRS classification faces three major barriers: limited datasets, confusing evaluation criteria, and domain barriers. We apply more appropriate evaluation methods to three open-access datasets to solve the first two barriers. For domain barriers, we propose a general and scalable vision fNIRS framework that converts multi-channel fNIRS signals into multi-channel virtual images using the Gramian angular difference field (GADF). We use the framework to train state-of-the-art visual models from computer vision (CV) within a few minutes, and the classification performance is competitive with the latest fNIRS models. In cross-validation experiments, visual models achieve the highest average classification results of 78.68% and 73.92% on mental arithmetic and word generation tasks, respectively. Although visual models are slightly lower than the fNIRS models on unilateral finger- and foot-tapping tasks, the F1-score and kappa coefficient indicate that these differences are insignificant in subject-independent experiments. Furthermore, we study fNIRS signal representations and the classification performance of sequence-to-image methods. We hope to introduce rich achievements from the CV domain to improve fNIRS classification research.

Identifying Individuals by fNIRS-Based Brain Functional Network Fingerprints

Individual identification based on brain functional network (BFN) has attracted a lot of research interest in recent years, since it provides a novel biometric for identity authentication, as well as a feasible way of exploring the brain at an individual level. Previous studies have shown that an individual can be identified by its BFN fingerprint estimated from functional magnetic resonance imaging, electroencephalogram, or magnetoencephalography data. Functional near-infrared spectroscopy (fNIRS) is an emerging imaging technique that, by measuring the changes in blood oxygen concentration, can respond to cerebral activities; in this paper, we investigate whether fNIRS-based BFN could be used as a “fingerprint” to identify individuals. In particular, Pearson's correlation is first used to calculate BFN based on the preprocessed fNIRS signals, and then the nearest neighbor scheme is used to match the estimated BFNs between different individuals. Through the experiments on an open-access fNIRS dataset, we have two main findings: (1) under the cases of cross-task (i.e., resting, right-handed, left-handed finger tapping, and foot tapping), the BFN fingerprints generally work well for the individual identification, and, more interestingly, (2) the accuracy under cross-task is well above the accuracy under cross-view (i.e., oxyhemoglobin and de-oxyhemoglobin). These findings indicate that fNIRS-based BFN fingerprint is a potential biometric for identifying individual.

Transformer Model for Functional Near-Infrared Spectroscopy Classification

Functional near-infrared spectroscopy (fNIRS) is a promising neuroimaging technology. The fNIRS classification problem has always been the focus of the brain-computer interface (BCI). Inspired by the success of Transformer based on self-attention mechanism in the fields of natural language processing and computer vision, we propose an fNIRS classification network based on Transformer, named fNIRS-T. We explore the spatial-level and channel-level representation of fNIRS signals to improve data utilization and network representation capacity. Besides, a preprocessing module, which consists of one-dimensional average pooling and layer normalization, is designed to replace filtering and baseline correction of data preprocessing. It makes fNIRS-T an end-to-end network, called fNIRS-PreT. Compared with traditional machine learning classifiers, convolutional neural network (CNN), and long short-term memory (LSTM), the proposed models obtain the best accuracy on three open-access datasets. Specifically, in the most extensive ternary classification task (30 subjects) that includes three types of overt movements, fNIRS-T, CNN, and LSTM obtain 75.49%, 72.89%, and 61.94% on test sets, respectively. Compared to traditional classifiers, fNIRS-T is at least 27.41% higher than statistical features and 6.79% higher than well-designed features. In the individual subject experiment of the ternary classification task, fNIRS-T achieves an average subject accuracy of 78.22% and surpasses CNN and LSTM by a large margin of +4.75% and +11.33%. fNIRS-PreT using raw data also achieves competitive performance to fNIRS-T. Therefore, the proposed models improve the performance of fNIRS-based BCI significantly.

Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e., optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky–Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.

LSTM based GAN Networks for Enhancing Ternary Task Classification Using fNIRS Data

Brain activation patterns vary according to the tasks performed by the subject. Neuroimaging techniques can be used to map the functioning of the cortex to capture brain activation patterns. Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique increasingly used for task classification based on brain activation patterns. fNIRS can be widely used in population studies due to the technology's economic,non-invasive, and portable nature. The multidimensional and complex nature of fNIRS data makes it ideal for deep learning algorithms for classification. Most deep learning algorithms need a large amount of data to be appropriately trained. Generative networks can be used in such cases where a substantial amount of data is required. Still, the collection is complex due to various constraints. Conditional Generative Adversarial Networks (CGAN) can generate artificial samples of a specific category to improve the deep learning classifier's accuracy when the sample size is insufficient. The proposed system uses an LSTM based CGAN with an LSTM classifier to enhance the accuracy through data augmentation. The system can determine whether the subject's task is a Left Finger Tap, Right Finger Tap, or Foot Tap based on the fNIRS data patterns. The authors obtained a task classification accuracy of 90.2% for the LSTM based GAN combination.Clinical relevance- Acquiring medical data present practical difficulties due to time, money, labor, and economic cost. The deep learning-based model can better perform medical image classification than hand-crafted features when dealing with many data. GAN-based networks can be valuable in the medical field where collecting extensive data is not feasible. GAN-generated synthetic data can be used to improve the classification accuracy of classification systems.

Conditional-GAN Based Data Augmentation for Deep Learning Task Classifier Improvement Using fNIRS Data

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique used for mapping the functioning human cortex. fNIRS can be widely used in population studies due to the technology’s economic, non-invasive, and portable nature. fNIRS can be used for task classification, a crucial part of functioning with Brain-Computer Interfaces (BCIs). fNIRS data are multidimensional and complex, making them ideal for deep learning algorithms for classification. Deep Learning classifiers typically need a large amount of data to be appropriately trained without over-fitting. Generative networks can be used in such cases where a substantial amount of data is required. Still, the collection is complex due to various constraints. Conditional Generative Adversarial Networks (CGAN) can generate artificial samples of a specific category to improve the accuracy of the deep learning classifier when the sample size is insufficient. The proposed system uses a CGAN with a CNN classifier to enhance the accuracy through data augmentation. The system can determine whether the subject’s task is a Left Finger Tap, Right Finger Tap, or Foot Tap based on the fNIRS data patterns. The authors obtained a task classification accuracy of 96.67% for the CGAN-CNN combination.

Dual stream neural networks for brain signal classification

Objective. The primary objective of this work is to develop a neural nework classifier for arbitrary collections of functional neuroimaging signals to be used in brain–computer interfaces (BCIs). Approach. We propose a dual stream neural network (DSNN) for the classification problem. The first stream is an end-to-end classifier taking raw time-dependent signals as input and generating feature identification signatures from them. The second stream enhances the identified features from the first stream by adjoining a dynamic functional connectivity matrix aimed at incorporating nuanced multi-channel information during specified BCI tasks. Main results. The proposed DSNN classifier is benchmarked against three publicly available datasets, where the classifier demonstrates performance comparable to, or better than the state-of-art in each instance. An information theoretic examination of the trained network is also performed, utilizing various tools, to demonstrate how to glean interpretive insight into how the hidden layers of the network parse the underlying biological signals. Significance . The resulting DSNN is a subject-independent classifier that works for any collection of 1D functional neuroimaging signals, with the option of integrating domain specific information in the design.

Prefrontal Asymmetry BCI Neurofeedback Datasets

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Effective brain connectivity for fNIRS data analysis based on multi-delays symbolic phase transfer entropy

OBJECTIVE

Recently, effective connectivity (EC) calculation methods for functional near-infrared spectroscopy (fNIRS) data mainly face two problems: the first problem is that noise can seriously affect the EC calculation and even lead to false connectivity; the second problem is that it ignores the various real neurotransmission delays between the brain region, and instead uses a fixed delay coefficient for calculation.

APPROACH

To overcome these two issues, a delay symbolic phase transfer entropy (dSPTE) is proposed by developing traditional transfer entropy (TE) to estimate EC for fNIRS. Firstly, the phase time sequence was obtained from the original sequence by the Hilbert transform and state-space reconstruction was realized using a uniform embedding scheme. Then, a symbolization technique was applied based on a neural-gas algorithm to improve its noise robustness. Finally, the EC was calculated on multiple time delay scales to match different inter-region neurotransmission delays.

MAIN RESULTS

A linear AR model, a nonlinear model and a multivariate hybrid model were introduced to simulate the performance of dSPTE, and the results showed that the accuracy of dSPTE was the highest, up to 74.27%, and specificity was 100% which means no false connectivity. The results confirmed that the dSPTE method realized better noise robustness, higher accuracy, and correct identification even if there was a long delay between series. Finally, we applied dSPTE to fNIRS dataset to analyse the EC during the finger-tapping task, the results showed that EC strength of task state significantly increased compared with the resting state.

SIGNIFICANCE

The proposed dSPTE method is a promising way to measure the EC for fNIRS. It incorporates the phase information TE with a symbolic process for fNIRS analysis for the first time. It has been confirmed to be noise robust and suitable for the complex network with different coupling delays.