fNIRS-based brain-computer interfaces: a review

Adaptive filter of frequency bands based coordinate attention network for EEG-based motor imagery classification

Purpose

In the brain-computer interface (BCI), motor imagery (MI) could be defined as the Electroencephalogram (EEG) signals through imagined movements, and ultimately enabling individuals to control external devices. However, the low signal-to-noise ratio, multiple channels and non-linearity are the essential challenges of accurate MI classification. To tackle these issues, we investigate the role of adaptive frequency bands selection and spatial-temporal feature learning in decoding motor imagery.

Methods

We propose an Adaptive Filter of Frequency Bands based Coordinate Attention Network (AFFB-CAN) to improve the performance of MI classification. Specifically, we design the AFFB to adaptively obtain the upper and lower limits of frequency bands in order to alleviate information loss caused by manual selection. Next, we propose the CAN-based network to emphasize the key brain regions and temporal segments. And we design a multi-scale module to enhance temporal context learning.

Results

The conducted experiments on the BCI Competition IV-2a and 2b datasets reveal that our approach achieves an outstanding average accuracy, kappa values, and Macro F1-Score with 0.7825, 0.7104, and 0.7486 respectively. Similarly, for the BCI Competition IV-2b dataset, the average accuracy, kappa values, and F1-Score obtained are 0.8879, 0.7427, and 0.8734, respectively.

Conclusion

The proposed AFFB-CAN method improves the performance of MI classification. In addition, our study confirms previous findings that motor imagery is mainly associated with µ and β rhythms. Moreover, we also find that γ rhythms also play an important role in MI classification.

Diagnostic machine learning applications on clinical populations using functional near infrared spectroscopy: a review

Functional near-infrared spectroscopy (fNIRS) and its interaction with machine learning (ML) is a popular research topic for the diagnostic classification of clinical disorders due to the lack of robust and objective biomarkers. This review provides an overview of research on psychiatric diseases by using fNIRS and ML. Article search was carried out and 45 studies were evaluated by considering their sample sizes, used features, ML methodology, and reported accuracy. To our best knowledge, this is the first review that reports diagnostic ML applications using fNIRS. We found that there has been an increasing trend to perform ML applications on fNIRS-based biomarker research since 2010. The most studied populations are schizophrenia (n = 12), attention deficit and hyperactivity disorder (n = 7), and autism spectrum disorder (n = 6) are the most studied populations. There is a significant negative correlation between sample size (>21) and accuracy values. Support vector machine (SVM) and deep learning (DL) approaches were the most popular classifier approaches (SVM = 20) (DL = 10). Eight of these studies recruited a number of participants more than 100 for classification. Concentration changes in oxy-hemoglobin (ΔHbO) based features were used more than concentration changes in deoxy-hemoglobin (ΔHb) based ones and the most popular ΔHbO-based features were mean ΔHbO (n = 11) and ΔHbO-based functional connections (n = 11). Using ML on fNIRS data might be a promising approach to reveal specific biomarkers for diagnostic classification.

Identifying Naturalistic Movies from Human Brain Activity with High-Density Diffuse Optical Tomography

Modern neuroimaging modalities, particularly functional MRI (fMRI), can decode detailed human experiences. Thousands of viewed images can be identified or classified, and sentences can be reconstructed. Decoding paradigms often leverage encoding models that reduce the stimulus space into a smaller yet generalizable feature set. However, the neuroimaging devices used for detailed decoding are non-portable, like fMRI, or invasive, like electrocorticography, excluding application in naturalistic use. Wearable, non-invasive, but lower-resolution devices such as electroencephalography and functional near-infrared spectroscopy (fNIRS) have been limited to decoding between stimuli used during training. Herein we develop and evaluate model-based decoding with high-density diffuse optical tomography (HD-DOT), a higher-resolution expansion of fNIRS with demonstrated promise as a surrogate for fMRI. Using a motion energy model of visual content, we decoded the identities of novel movie clips outside the training set with accuracy far above chance for single-trial decoding. Decoding was robust to modulations of testing time window, different training and test imaging sessions, hemodynamic contrast, and optode array density. Our results suggest that HD-DOT can translate detailed decoding into naturalistic use.

Cortical signals analysis to recognize intralimb mobility using modified RNN and various EEG quantities

Electroencephalogram (EEG) signals are critical in interpreting sensorimotor activities for predicting body movements. However, their efficacy in identifying intralimb movements, such as the dorsiflexion and plantar flexion of the foot, remains suboptimal. This study aims to explore whether various EEG signal quantities can effectively recognize intralimb movements to facilitate the development of Brain-Computer Interface (BCI) devices for foot rehabilitation. This research involved twenty-two healthy, right-handed participants. EEG data were collected using 21 electrodes positioned over the motor cortex, while two electromyography (EMG) electrodes recorded the onset of ankle joint movements. The study focused on analyzing slow cortical potential (SCP) and sensorimotor rhythms (SMR) in alpha and beta bands from the EEG. Five key features-fourth-order Autoregressive feature, variance, waveform length, standard deviation, and permutation entropy-were extracted. A modified Recurrent Neural Network (RNN) including Long Short-term Memory (LSTM) and Gated Recurrent Unit (GRU) algorithms was developed for movement recognition. These were compared against conventional machine learning algorithms, including nonlinear Support Vector Machine (SVM) and k Nearest Neighbourhood (kNN) classifiers. The performance of the proposed models was assessed using two data schemes: within-subject and across-subjects. The findings demonstrated that the GRU and LSTM models significantly outperformed traditional machine learning algorithms in recognizing different EEG signal quantities for intralimb movement. The study indicates that deep learning models, particularly GRU and LSTM, hold superior potential over standard machine learning techniques in identifying intralimb movements using EEG signals. Where the accuracies of LSTM for within and across subjects were 98.87 ± 1.80 % and 87.38 ± 0.86 % respectively. Whereas the accuracy of GRU within and across subjects were 99.18 ± 1.28 % and 86.44 ± 0.69 % respectively. This advancement could significantly benefit the development of BCI devices aimed at foot rehabilitation, suggesting a new avenue for enhancing physical therapy outcomes.

Working memory load recognition with deep learning time series classification

Working memory load (WML) is one of the widely applied signals in the areas of human-machine interaction. The precise evaluation of the WML is crucial for this kind of application. This study aims to propose a deep learning (DL) time series classification (TSC) model for inter-subject WML decoding. We used fNIRS to record the hemodynamic signals of 27 participants during visual working memory tasks. Traditional machine learning and deep time series classification algorithms were respectively used for intra-subject and inter-subject WML decoding from the collected blood oxygen signals. The intra-subject classification accuracy of LDA and SVM were 94.6% and 79.1%. Our proposed TAResnet-BiLSTM model had the highest inter-subject WML decoding accuracy, reaching 92.4%. This study provides a new idea and method for the brain-computer interface application of fNIRS in real-time WML detection.

The efficacy and methodology of using near-infrared spectroscopy to determine resting-state brain networks

Functional connectivity is a critical aspect of brain function and is essential for understanding, diagnosing, and treating neurological and psychiatric disorders. It refers to the synchronous activity between different regions of the brain, which gives rise to communication and information processing. Resting-state functional connectivity is a subarea of study that allows researchers to examine brain activity in the absence of a task or stimulus. This can provide insight into the brain's intrinsic functional architecture and help identify neural networks that are active during rest. Thus, determining functional connectivity topography is valuable both clinically and in research. Traditional methods using functional magnetic resonance imaging have proven to be effective, however, they have their limitations. In this review, we investigate the feasibility of using functional near-infrared spectroscopy (fNIRS) as a low-cost, portable alternative for measuring functional connectivity. We first establish fNIRS' ability to detect localized brain activity during task-based experiments. Next, we verify its use in resting-state studies with results showing a high degree of correspondence with resting-state functional magnetic resonance imaging (rs-fMRI). Also discussed are various data-processing methods and the validity of filtering the global signal, which is the current standard for analysis. We consider the possible origins of the global signal, if it contains pertinent neuronal information that could be of importance in better understanding neuronal networks, and what we believe is the best method of approaching signal analysis and regression.

Alterations in brain functional connectivity in patients with mild cognitive impairment: A systematic review and meta-analysis of functional near-infrared spectroscopy studies

Emerging evidences suggest that cognitive deficits in individuals with mild cognitive impairment (MCI) are associated with disruptions in brain functional connectivity (FC). This systematic review and meta-analysis aimed to comprehensively evaluate alterations in FC between MCI individuals and healthy control (HC) using functional near-infrared spectroscopy (fNIRS). Thirteen studies were included in qualitative analysis, with two studies synthesized for quantitative meta-analysis. Overall, MCI patients exhibited reduced resting-state FC, predominantly in the prefrontal, parietal, and occipital cortex. Meta-analysis of two studies revealed a significant reduction in resting-state FC from the right prefrontal to right occipital cortex (standardized mean difference [SMD] = -.56; p < .001), left prefrontal to left occipital cortex (SMD = -.68; p < .001), and right prefrontal to left occipital cortex (SMD = -.53; p < .001) in MCI patients compared to HC. During naming animal-walking task, MCI patients exhibited enhanced FC in the prefrontal, motor, and occipital cortex, whereas a decrease in FC was observed in the right prefrontal to left prefrontal cortex during calculating-walking task. In working memory tasks, MCI predominantly showed increased FC in the medial and left prefrontal cortex. However, a decreased in prefrontal FC and a shifted in distribution from the left to the right prefrontal cortex were noted in MCI patients during a verbal frequency task. In conclusion, fNIRS effectively identified abnormalities in FC between MCI and HC, indicating disrupted FC as potential markers for the early detection of MCI. Future studies should investigate the use of task- and region-specific FC alterations as a sensitive biomarker for MCI.

Subject-specific information enhances spatial accuracy of high-density diffuse optical tomography

Functional near-infrared spectroscopy (fNIRS) is a widely used imaging method for mapping brain activation based on cerebral hemodynamics. The accurate quantification of cortical activation using fNIRS data is highly dependent on the ability to correctly localize the positions of light sources and photodetectors on the scalp surface. Variations in head size and shape across participants greatly impact the precise locations of these optodes and consequently, the regions of the cortical surface being reached. Such variations can therefore influence the conclusions drawn in NIRS studies that attempt to explore specific cortical regions. In order to preserve the spatial identity of each NIRS channel, subject-specific differences in NIRS array registration must be considered. Using high-density diffuse optical tomography (HD-DOT), we have demonstrated the inter-subject variability of the same HD-DOT array applied to ten participants recorded in the resting state. We have also compared three-dimensional image reconstruction results obtained using subject-specific positioning information to those obtained using generic optode locations. To mitigate the error introduced by using generic information for all participants, photogrammetry was used to identify specific optode locations per-participant. The present work demonstrates the large variation between subjects in terms of which cortical parcels are sampled by equivalent channels in the HD-DOT array. In particular, motor cortex recordings suffered from the largest optode localization errors, with a median localization error of 27.4 mm between generic and subject-specific optodes, leading to large differences in parcel sensitivity. These results illustrate the importance of collecting subject-specific optode locations for all wearable NIRS experiments, in order to perform accurate group-level analysis using cortical parcellation.

Unleashing the potential of fNIRS with machine learning: classification of fine anatomical movements to empower future brain-computer interface

In this study, we explore the potential of using functional near-infrared spectroscopy (fNIRS) signals in conjunction with modern machine-learning techniques to classify specific anatomical movements to increase the number of control commands for a possible fNIRS-based brain-computer interface (BCI) applications. The study focuses on novel individual finger-tapping, a well-known task in fNIRS and fMRI studies, but limited to left/right or few fingers. Twenty-four right-handed participants performed the individual finger-tapping task. Data were recorded by using sixteen sources and detectors placed over the motor cortex according to the 10-10 international system. The event's average oxygenated Δ HbO and deoxygenated Δ HbR hemoglobin data were utilized as features to assess the performance of diverse machine learning (ML) models in a challenging multi-class classification setting. These methods include LDA, QDA, MNLR, XGBoost, and RF. A new DL-based model named "Hemo-Net" has been proposed which consists of multiple parallel convolution layers with different filters to extract the features. This paper aims to explore the efficacy of using fNRIS along with ML/DL methods in a multi-class classification task. Complex models like RF, XGBoost, and Hemo-Net produce relatively higher test set accuracy when compared to LDA, MNLR, and QDA. Hemo-Net has depicted a superior performance achieving the highest test set accuracy of 76%, however, in this work, we do not aim at improving the accuracies of models rather we are interested in exploring if fNIRS has the neural signatures to help modern ML/DL methods in multi-class classification which can lead to applications like brain-computer interfaces. Multi-class classification of fine anatomical movements, such as individual finger movements, is difficult to classify with fNIRS data. Traditional ML models like MNLR and LDA show inferior performance compared to the ensemble-based methods of RF and XGBoost. DL-based method Hemo-Net outperforms all methods evaluated in this study and demonstrates a promising future for fNIRS-based BCI applications.

fNIRS Dataset During Complex Scene Analysis

When analyzing complex scenes, humans often focus their attention on an object at a particular spatial location. The ability to decode the attended spatial location would facilitate brain computer interfaces for complex scene analysis (CSA). Here, we investigated capability of functional near-infrared spectroscopy (fNIRS) to decode audio-visual spatial attention in the presence of competing stimuli from multiple locations. We targeted dorsal frontoparietal network including frontal eye field (FEF) and intra-parietal sulcus (IPS) as well as superior temporal gyrus/planum temporal (STG/PT). They all were shown in previous functional magnetic resonance imaging (fMRI) studies to be activated by auditory, visual, or audio-visual spatial tasks. To date, fNIRS has not been applied to decode auditory and visual-spatial attention during CSA, and thus, no such dataset exists yet. This report provides an open-access fNIRS dataset that can be used to develop, test, and compare machine learning algorithms for classifying attended locations based on the fNIRS signals on a single trial basis.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

Technical note: High-efficient and wireless transcranial ultrasound excitation based on electromagnetic acoustic transducer

BACKGROUND

The generation of transcranial ultrasound is usually based on the piezoelectric effect, so it is necessary to attach transducers around the skull. However, the skull will cause serious attenuation and scattering of ultrasound, which makes it particularly difficult for transcranial ultrasound imaging and modulation.

PURPOSE

In transcranial ultrasound imaging, there is significant attenuation and scattering of ultrasound waves by the skull bone. To mitigate this influence and enable precise imaging and high-efficient transcranial ultrasound for specific patients (such as stroke patients who already require craniotomy as part of their surgical care), this paper proposes to use EMAT to excite metal plates placed inside the skull based on the excellent penetration characteristics of EM waves into the skull, generating ultrasound signals, which can completely avoid the influence of skull on ultrasound transmission.

METHODS

Based on an efficient wireless transcranial ultrasound experimental platform, we first verified that the skull would not affect the propagation of electromagnetic waves generated by EMAT. In addition, the distribution of the transcranial sound field generated by EMAT was measured.

RESULTS

EMAT can generate 1.0 MHz ultrasound by wireless excitation of a 0.1 mm thick copper plate through an adult skull with a thickness of ∼1 cm, and the frequency and amplitude of the generated ultrasound are not affected by the skull. The results indicated that the electromagnetic waves successfully penetrated the skull, with a recorded strength of approximately 2 mV. We also found that the ultrasound signals generated by the EMAT probe through the skull remained unaffected, measuring around 2 mV. In addition, the measurement of the transcranial sound field distribution (80*50 mm2 ) generated by EMAT shows that compared with the traditional extracranial ultrasound generation method, the sound field distribution generated by the wireless excitation of the intracranial copper plate based on EAMT is no longer affected by the uneven and irregular skull.

CONCLUSION

Our experiments involved validating the penetration capabilities of electromagnetic waves utilizing the EMAT probe through a 7 (5+2) mm thick organic glass plate and a real human skull ranging from 8 to 15 mm in thickness. The efficient and wireless transcranial ultrasound excitation proposed in this paper may be possible for transcranial ultrasound imaging and therapy.

Machines, mathematics, and modules: the potential to provide real-time metrics for pain under anesthesia

The brain-based assessments under anesthesia have provided the ability to evaluate pain/nociception during surgery and the potential to prevent long-term evolution of chronic pain. Prior studies have shown that the functional near-infrared spectroscopy (fNIRS)-measured changes in cortical regions such as the primary somatosensory and the polar frontal cortices show consistent response to evoked and ongoing pain in awake, sedated, and anesthetized patients. We take this basic approach and integrate it into a potential framework that could provide real-time measures of pain/nociception during the peri-surgical period. This application could have significant implications for providing analgesia during surgery, a practice that currently lacks quantitative evidence to guide patient tailored pain management. Through a simple readout of "pain" or "no pain," the proposed system could diminish or eliminate levels of intraoperative, early post-operative, and potentially, the transition to chronic post-surgical pain. The system, when validated, could also be applied to measures of analgesic efficacy in the clinic.

Learning based motion artifacts processing in fNIRS: a mini review

This paper provides a concise review of learning-based motion artifacts (MA) processing methods in functional near-infrared spectroscopy (fNIRS), highlighting the challenges of maintaining optimal contact during subject movement, which can lead to MA and compromise data integrity. Traditional strategies often result in reduced reliability of the hemodynamic response and statistical power. Recognizing the limited number of studies focusing on learning-based MA removal, we examine 315 studies, identifying seven pertinent to our focus area. We discuss the current landscape of learning-based MA correction methods and highlight research gaps. Noting the absence of standard evaluation metrics for quality assessment of MA correction, we suggest a novel framework, integrating signal and model quality considerations and employing metrics like ΔSignal-to-Noise Ratio (ΔSNR), confusion matrix, and Mean Squared Error. This work aims to facilitate the application of learning-based methodologies to fNIRS and improve the accuracy and reliability of neurovascular studies.

Feasibility of local interpretable model-agnostic explanations (LIME) algorithm as an effective and interpretable feature selection method: comparative fNIRS study

Many feature selection methods have been evaluated in functional near-infrared spectroscopy (fNIRS)-related studies. The local interpretable model-agnostic explanation (LIME) algorithm is a feature selection method for fNIRS datasets that has not yet been validated; the demand for its validation is increasing. To this end, we assessed the feature selection performance of LIME for fNIRS datasets in terms of classification accuracy. A comparative analysis was conducted for the benchmark (classification accuracy obtained without applying any feature selection method), LIME, two filter-based methods (minimum-redundancy maximum-relevance and t-test), and one wrapper-based method (sequential forward selection). To ensure the fairness and reliability of the performance evaluation, several open-access fNIRS datasets were used. The analysis revealed that LIME greatly outperformed the other feature selection methods in most cases and could achieve a statistically significantly better classification accuracy than that of the benchmark methods. These findings implied the effectiveness of LIME as a feature selection approach for fNIRS datasets.

Deep neural network to differentiate internet gaming disorder from healthy controls during stop-signal task: a multichannel near-infrared spectroscopy study

Internet Gaming Disorder (IGD), as one of worldwide mental health issues, leads to negative effects on physical and mental health and has attracted public attention. Most studies on IGD are based on screening scales and subjective judgments of doctors, without objective quantitative assessment. However, public understanding of internet gaming disorder lacks objectivity. Therefore, the researches on internet gaming disorder still have many limitations. In this paper, a stop-signal task (SST) was designed to assess inhibitory control in patients with IGD based on prefrontal functional near-infrared spectroscopy (fNIRS). According to the scale, the subjects were divided into health and gaming disorder. A total of 40 subjects (24 internet gaming disorders; 16 healthy controls) signals were used for deep learning-based classification. The seven algorithms used for classification and comparison were deep learning algorithms (DL) and machine learning algorithms (ML), with four and three algorithms in each category, respectively. After applying hold-out method, the performance of the model was verified by accuracy. DL models outperformed traditional ML algorithms. Furthermore, the classification accuracy of the two-dimensional convolution neural network (2D-CNN) was 87.5% among all models. This was the highest accuracy out of all models that were tested. The 2D-CNN was able to outperform the other models due to its ability to learn complex patterns in data. This makes it well-suited for image classification tasks. The findings suggested that a 2D-CNN model is an effective approach for predicting internet gaming disorder. The results show that this is a reliable method with high accuracy to identify patients with IGD and demonstrate that the use of fNIRS to facilitate the development of IGD diagnosis has great potential.

Deep Learning Based Walking Tasks Classification in Older Adults Using fNIRS

Decline in gait features is common in older adults and an indicator of increased risk of disability, morbidity, and mortality. Under dual task walking (DTW) conditions, further degradation in the performance of both the gait and the secondary cognitive task were found in older adults which were significantly correlated to falls history. Cortical control of gait, specifically in the pre-frontal cortex (PFC) as measured by functional near infrared spectroscopy (fNIRS), during DTW in older adults has recently been studied. However, the automatic classification of differences in cognitive activations under single and dual task gait conditions has not been extensively studied yet. In this paper, by considering single task walking (STW) as a lower attentional walking state and DTW as a higher attentional walking state, we aimed to formulate this as an automatic detection of low and high attentional walking states and leverage deep learning methods to perform their classification. We conduct analysis on the data samples which reveals the characteristics on the difference between HbO2 and Hb values that are subsequently used as additional features. We perform feature engineering to formulate the fNIRS features as a 3-channel image and apply various image processing techniques for data augmentation to enhance the performance of deep learning models. Experimental results show that pre-trained deep learning models that are fine-tuned using the collected fNIRS dataset together with gender and cognitive status information can achieve around 81% classification accuracy which is about 10% higher than the traditional machine learning algorithms. We present additional sensitivity metrics such as confusion matrix, precision and F1 score, as well as accuracy on two-way classification between condition pairings. We further performed an extensive ablation study to evaluate factors such as the voxel locations, channels of input images, zero-paddings and pre-training of deep learning model on their contribution or impact to the classification task. Results showed that using pre-trained model, all the voxel locations, and HbO2 - Hb as the third channel of the input image can achieve the best classification accuracy.

Hierarchical Classification Strategy for Mitigating the Impact of The Presence of Pain in fNIRS-based BCIs

Brain computer interfaces (BCIs) can find applications in assistive systems for patients who experience conditions that impede their motor abilities. A BCI uses signals acquired from the brain to control external devices. As physical pain influences cortical signals, the presence of pain can negatively impact the performance of the BCI. In this work, we propose a strategy to mitigate this negative impact. Cortical signals are acquired from test subjects while they performed two mental arithmetic tasks, in the presence and the absence of painful stimuli. The task of the BCI is to reliably classify the two mental arithmetic tasks from the cortical recordings, irrespective of the presence or the absence of pain. We propose to do this classification, hierarchically, in two levels. In the first level, the data is classified into those captured in the presence and the absence of pain. Depending on the results of the classification from the first level, in the second level, the BCI performs the classification of tasks using a classifier trained either in the presence or the absence of pain. A 1-dimensional convolutional neural network (1D-CNN) is used for classification at both levels. It is observed that using this hierarchical strategy, the BCI is able to classify the tasks with an accuracy greater than 90%, irrespective of the presence or the absence of pain. Given that the presence of physical pain has shown previously to reduce the classification accuracy of a BCI to almost chance levels, this mitigation strategy will be a significant step towards enhancing the performance of BCIs when they are used in assistive systems for patients.

The Effects of Different Brain Regions on fNIRS-based Task-state Detection in Speech Imagery

Brain-computer interface (BCI) based on speech imagery can decode users' verbal intent and help people with motor disabilities communicate naturally. Functional near-infrared spectroscopy (fNIRS) is a commonly used brain signal acquisition method. Asynchronous BCI can response to control commands at any time, which provides great convenience for users. Task state detection, defined as identifying whether user starts or continues covertly articulating, plays an important role in speech imagery BCIs. To better distinguish task state from idle state during speech imagery, this work used fNIRS signals from different brain regions to study the effects of different brain regions on task state detection accuracy. The imagined tonal syllables included four lexical tones and four vowels in Mandarin Chinese. The brain regions that were measured included Broca's area, Wernicke's area, Superior temporal cortex and Motor cortex. Task state detection accuracies of imagining tonal monosyllables with four different tones were analyzed. The average accuracy of four speech imagery tasks based on the whole brain was 0.67 and it was close to 0.69, which was the average accuracy based on Broca's area. The accuracies of Broca's area and the whole brain were significantly higher than those of other brain regions. The findings of this work demonstrated that using a few channels of Broca's area could result in a similar task state detection accuracy to that using all the channels of the brain. Moreover, it was discovered that speech imagery with tone 2/3 tasks yielded higher task state detection accuracy than speech imagery with other tones.

Decoupling brain activations of muscle-caused activations and mental intention-cause activations using the general linear model: A functional near-infrared spectroscopy study

Integrating a brain-computer interface into a lower-limb medical rehabilitation assistive device can enhance rehabilitation efficiency. The latest research in the field focuses on the decoding performance of different motions. However, the difference between muscle-caused primitive activation and mental intention-caused activation has not been fully investigated. Thus, our study tried to decouple these two kinds of cerebral activation using a general linear model (GLM). Nine healthy and right-handed subjects were recruited for a two-section experiment. They were asked to extend or flex their knees while seated in the first section of the experiment or standing in the second section. Functional near-infrared spectroscopy (fNIR) was adopted to monitor their hemodynamic changes. Two groups of paradigms (one for circle-wise analysis, the other for full-section analysis) were constructed from the experimental paradigm. Each group consisted of three (the first intention, the second intention, and the muscle activation). The constructed paradigms were fed to the Balloon model for six desired hemodynamic responses (dHRFs). The regressor of GLM consisted of three dHRFs and the corresponding motion artifacts and drifts. The simulated physiological noises were included in the structured background matrix. The results showed that all subjects had similar cerebral activation patterns for the intention to extend or flex knees. The activation during musclecaused activation was less intense than that caused by both intentions. This finding can help further research on more efficient motion intention detection and the possibility of multiple motions decoding.

Transformer Based Cross-Subject Mental Workload Classification Using FNIRS for Real-World Application

Mental state monitoring is a hot topic especially in neurorehabilitation, skill training, etc, for which the functional near-infrared spectroscopy (fNIRS) has been suggested to be used, and fewer detection channels and cross-subject performance are usually required for real-world application. To this goal, we propose a transformer-based method for cross-subject mental workload classification using fewer channels of fNIRS. Firstly, the input fNIRS signals in a window are divided into patches in the temporal order and transformed into embeddings, to which a classification token and learnable position embeddings are added. Then, a transformer encoder is used to learn the long-range dependencies among the embeddings, of which the output classification token is sent to a multilayer perceptron (MLP) head. Mental workload classification results can be represented by the outputs of the MLP head. Finally, comparison experiments were conducted on the open-access fNIRS2MW dataset. The results show that, the proposed method can outperform previous methods in cross-subject classification accuracy, and relatively efficient computation can be obtained.

Scalable, modular continuous wave functional near-infrared spectroscopy system (Spotlight)

Significance

We present a fiberless, portable, and modular continuous wave-functional near-infrared spectroscopy system, Spotlight, consisting of multiple palm-sized modules-each containing high-density light-emitting diode and silicon photomultiplier detector arrays embedded in a flexible membrane that facilitates optode coupling to scalp curvature.

Aim

Spotlight's goal is to be a more portable, accessible, and powerful functional near-infrared spectroscopy (fNIRS) device for neuroscience and brain-computer interface (BCI) applications. We hope that the Spotlight designs we share here can spur more advances in fNIRS technology and better enable future non-invasive neuroscience and BCI research.

Approach

We report sensor characteristics in system validation on phantoms and motor cortical hemodynamic responses in a human finger-tapping experiment, where subjects wore custom 3D-printed caps with two sensor modules.

Results

The task conditions can be decoded offline with a median accuracy of 69.6%, reaching 94.7% for the best subject, and at a comparable accuracy in real time for a subset of subjects. We quantified how well the custom caps fitted to each subject and observed that better fit leads to more observed task-dependent hemodynamic response and better decoding accuracy.

Conclusions

The advances presented here should serve to make fNIRS more accessible for BCI applications.

Measuring cognitively demanding activities in pediatric out-of-hospital cardiac arrest

BACKGROUND

This methodological intersection article demonstrates a method to measure cognitive load in clinical simulations. Researchers have hypothesized that high levels of cognitive load reduce performance and increase errors. This phenomenon has been studied primarily by experimental designs that measure responses to predetermined stimuli and self-reports that reduce the experience to a summative value. Our goal was to develop a method to identify clinical activities with high cognitive burden using physiologic measures.

METHODS

Teams of emergency medical responders were recruited from local fire departments to participate in a scenario with a shockable pediatric out-of-hospital cardiac arrest (POHCA) patient. The scenario was standardized with the patient being resuscitated after receiving high-quality CPR and 3 defibrillations. Each team had a person in charge (PIC) who wore a functional near-infrared spectroscopy (fNIRS) device that recorded changes in oxygenated and deoxygenated hemoglobin concentration in their prefrontal cortex (PFC), which was interpreted as cognitive activity. We developed a data processing pipeline to remove nonneural noise (e.g., motion artifacts, heart rate, respiration, and blood pressure) and detect statistically significant changes in cognitive activity. Two researchers independently watched videos and coded clinical tasks corresponding to detected events. Disagreements were resolved through consensus, and results were validated by clinicians.

RESULTS

We conducted 18 simulations with 122 participants. Participants arrived in teams of 4 to 7 members, including one PIC. We recorded the PIC's fNIRS signals and identified 173 events associated with increased cognitive activity. [Defibrillation] (N = 34); [medication] dosing (N = 33); and [rhythm checks] (N = 28) coincided most frequently with detected elevations in cognitive activity. [Defibrillations] had affinity with the right PFC, while [medication] dosing and [rhythm checks] had affinity with the left PFC.

CONCLUSIONS

FNIRS is a promising tool for physiologically measuring cognitive load. We describe a novel approach to scan the signal for statistically significant events with no a priori assumptions of when they occur. The events corresponded to key resuscitation tasks and appeared to be specific to the type of task based on activated regions in the PFC. Identifying and understanding the clinical tasks that require high cognitive load can suggest targets for interventions to decrease cognitive load and errors in care.

Translational opportunities and challenges of invasive electrodes for neural interfaces

Invasive brain-machine interfaces can restore motor, sensory and cognitive functions. However, their clinical adoption has been hindered by the surgical risk of implantation and by suboptimal long-term reliability. In this Review, we highlight the opportunities and challenges of invasive technology for clinically relevant electrophysiology. Specifically, we discuss the characteristics of neural probes that are most likely to facilitate the clinical translation of invasive neural interfaces, describe the neural signals that can be acquired or produced by intracranial electrodes, the abiotic and biotic factors that contribute to their failure, and emerging neural-interface architectures.

Metaheuristic Optimization-Based Feature Selection for Imagery and Arithmetic Tasks: An fNIRS Study

In recent decades, the brain-computer interface (BCI) has emerged as a leading area of research. The feature selection is vital to reduce the dataset's dimensionality, increase the computing effectiveness, and enhance the BCI's performance. Using activity-related features leads to a high classification rate among the desired tasks. This study presents a wrapper-based metaheuristic feature selection framework for BCI applications using functional near-infrared spectroscopy (fNIRS). Here, the temporal statistical features (i.e., the mean, slope, maximum, skewness, and kurtosis) were computed from all the available channels to form a training vector. Seven metaheuristic optimization algorithms were tested for their classification performance using a k-nearest neighbor-based cost function: particle swarm optimization, cuckoo search optimization, the firefly algorithm, the bat algorithm, flower pollination optimization, whale optimization, and grey wolf optimization (GWO). The presented approach was validated based on an available online dataset of motor imagery (MI) and mental arithmetic (MA) tasks from 29 healthy subjects. The results showed that the classification accuracy was significantly improved by utilizing the features selected from the metaheuristic optimization algorithms relative to those obtained from the full set of features. All of the abovementioned metaheuristic algorithms improved the classification accuracy and reduced the feature vector size. The GWO yielded the highest average classification rates (p < 0.01) of 94.83 ± 5.5%, 92.57 ± 6.9%, and 85.66 ± 7.3% for the MA, MI, and four-class (left- and right-hand MI, MA, and baseline) tasks, respectively. The presented framework may be helpful in the training phase for selecting the appropriate features for robust fNIRS-based BCI applications.

Evaluation of residual cognition in patients with disorders of consciousness based on functional near-infrared spectroscopy

SIGNIFICANCE

Accurate evaluation of consciousness in patients with prolonged disorders of consciousness (DOC) is critical for designing therapeutic plans, determining rehabilitative services, and predicting prognosis. Effective ways for detecting consciousness in patients with DOC are still needed.

AIM

Evaluation of the residual awareness in patients with DOC and investigation of the spatiotemporal differences in the hemodynamic responses between the minimally conscious state (MCS) and the unresponsive wakefulness syndrome (UWS) groups using active command-driven motor imagery (MI) tasks.

APPROACH

In this study, functional near-infrared spectroscopy (fNIRS) was used to measure the changes of hemodynamic responses in 19 patients with DOC (9 MCS and 10 UWS) using active command-driven MI tasks. The characteristics of the hemodynamic responses were extracted to compare the differences between the MCS and UWS groups. Moreover, the correlations between the hemodynamic responses and the clinical behavioral evaluations were also studied.

RESULTS

The results showed significant differences in the spatiotemporal distribution of the hemodynamic responses between the MCS and UWS groups. For the patients with MCS, significant increases in task-evoked hemodynamic responses occurred during the "YES" questions of the command-driven MI tasks. Importantly, these changes were significantly correlated with their coma-recovery scale-revised (CRS-R) scores. However, for the patients with UWS, no significant changes of the hemodynamic responses were found. Additionally, the results did not show any statistical correlation between the hemodynamic responses and their CRS-R scores.

CONCLUSIONS

The fNIRS-based command-driven MI tasks can be used as a promising tool for detecting residual awareness in patients with DOC. We hope that the findings and the active paradigm used in this study will provide useful insights into the diagnosis, therapy, and prognosis of this challenging patient population.

Using preregistration as a tool for transparent fNIRS study design

Significance

The expansion of functional near-infrared spectroscopy (fNIRS) methodology and analysis tools gives rise to various design and analytical decisions that researchers have to make. Several recent efforts have developed guidelines for preprocessing, analyzing, and reporting practices. For the planning stage of fNIRS studies, similar guidance is desirable. Study preregistration helps researchers to transparently document study protocols before conducting the study, including materials, methods, and analyses, and thus, others to verify, understand, and reproduce a study. Preregistration can thus serve as a useful tool for transparent, careful, and comprehensive fNIRS study design.

Aim

We aim to create a guide on the design and analysis steps involved in fNIRS studies and to provide a preregistration template specified for fNIRS studies.

Approach

The presented preregistration guide has a strong focus on fNIRS specific requirements, and the associated template provides examples based on continuous-wave (CW) fNIRS studies conducted in humans. These can, however, be extended to other types of fNIRS studies.

Results

On a step-by-step basis, we walk the fNIRS user through key methodological and analysis-related aspects central to a comprehensive fNIRS study design. These include items specific to the design of CW, task-based fNIRS studies, but also sections that are of general importance, including an in-depth elaboration on sample size planning.

Conclusions

Our guide introduces these open science tools to the fNIRS community, providing researchers with an overview of key design aspects and specification recommendations for comprehensive study planning. As such it can be used as a template to preregister fNIRS studies or merely as a tool for transparent fNIRS study design.

System Derived Spatial-Temporal CNN for High-Density fNIRS BCI

An intuitive and generalisable approach to spatial-temporal feature extraction for high-density (HD) functional Near-Infrared Spectroscopy (fNIRS) brain-computer interface (BCI) is proposed, demonstrated here using Frequency-Domain (FD) fNIRS for motor-task classification. Enabled by the HD probe design, layered topographical maps of Oxy/deOxy Haemoglobin changes are used to train a 3D convolutional neural network (CNN), enabling simultaneous extraction of spatial and temporal features. The proposed spatial-temporal CNN is shown to effectively exploit the spatial relationships in HD fNIRS measurements to improve the classification of the functional haemodynamic response, achieving an average F1 score of 0.69 across seven subjects in a mixed subjects training scheme, and improving subject-independent classification as compared to a standard temporal CNN.

Benchmarking framework for machine learning classification from fNIRS data

Background

While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods

We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion

Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.

Impacts of simplifying articulation movements imagery to speech imagery BCI performance

Objective.Speech imagery (SI) can be used as a reliable, natural, and user-friendly activation task for the development of brain-computer interface (BCI), which empowers individuals with severe disabilities to interact with their environment. The functional near-infrared spectroscopy (fNIRS) is advanced as one of the most suitable brain imaging methods for developing BCI systems owing to its advantages of being non-invasive, portable, insensitive to motion artifacts, and having relatively high spatial resolution.Approach.To improve the classification performance of SI BCI based on fNIRS, a novel paradigm was developed in this work by simplifying the articulation movements in SI to make the articulation movement differences clearer between different words imagery tasks. A SI BCI was proposed to directly answer questions by covertly rehearsing the word '' or '' ('yes' or 'no' in English), and an unconstrained rest task also was contained in this BCI. The articulation movements of SI were simplified by retaining only the movements of the jaw and lips of vowels in Chinese Pinyin for words '' and ''.Main results.Compared with conventional speech imagery, simplifying the articulation movements in SI could generate more different brain activities among different tasks, which led to more differentiable temporal features and significantly higher classification performance. The average 3-class classification accuracies of the proposed paradigm across all 20 participants reached 69.6% and 60.2% which were about 10.8% and 5.6% significantly higher than those of the conventional SI paradigm operated in the 0-10 s and 0-2.5 s time windows, respectively.Significance.These results suggested that simplifying the articulation movements in SI is promising for improving the classification performance of intuitive BCIs based on speech imagery.

Explainable artificial intelligence model to predict brain states from fNIRS signals

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode's output onto the input variables for fNIRS signals is described here. Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model's output in terms of the model's input. Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations. Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

Network organization of resting-state cerebral hemodynamics and their aliasing contributions measured by functional near-infrared spectroscopy

Objective. Spontaneous fluctuations of cerebral hemodynamics measured by functional magnetic resonance imaging (fMRI) are widely used to study the network organization of the brain. The temporal correlations among the ultra-slow, <0.1 Hz fluctuations across the brain regions are interpreted as functional connectivity maps and used for diagnostics of neurological disorders. However, despite the interest narrowed in the ultra-slow fluctuations, hemodynamic activity that exists beyond the ultra-slow frequency range could contribute to the functional connectivity, which remains unclear.Approach. In the present study, we have measured the brain-wide hemodynamics in the human participants with functional near-infrared spectroscopy (fNIRS) in a whole-head, cap-based and high-density montage at a sampling rate of 6.25 Hz. In addition, we have acquired resting state fMRI scans in the same group of participants for cross-modal evaluation of the connectivity maps. Then fNIRS data were deliberately down-sampled to a typical fMRI sampling rate of ∼0.5 Hz and the resulted differential connectivity maps were subject to a k-means clustering.Main results. Our diffuse optical topographical analysis of fNIRS data have revealed a default mode network (DMN) in the spontaneous deoxygenated and oxygenated hemoglobin changes, which remarkably resemble the same fMRI network derived from participants. Moreover, we have shown that the aliased activities in the down-sampled optical signals have altered the connectivity patterns, resulting in a network organization of aliased functional connectivity in the cerebral hemodynamics.Significance.The results have for the first time demonstrated that fNIRS as a broadly accessible modality can image the resting-state functional connectivity in the posterior midline, prefrontal and parietal structures of the DMN in the human brain, in a consistent pattern with fMRI. Further empowered by the fast sampling rate of fNIRS, our findings suggest the presence of aliased connectivity in the current understanding of the human brain organization.

Machine learning in biosignals processing for mental health: A narrative review

Machine Learning (ML) offers unique and powerful tools for mental health practitioners to improve evidence-based psychological interventions and diagnoses. Indeed, by detecting and analyzing different biosignals, it is possible to differentiate between typical and atypical functioning and to achieve a high level of personalization across all phases of mental health care. This narrative review is aimed at presenting a comprehensive overview of how ML algorithms can be used to infer the psychological states from biosignals. After that, key examples of how they can be used in mental health clinical activity and research are illustrated. A description of the biosignals typically used to infer cognitive and emotional correlates (e.g., EEG and ECG), will be provided, alongside their application in Diagnostic Precision Medicine, Affective Computing, and brain-computer Interfaces. The contents will then focus on challenges and research questions related to ML applied to mental health and biosignals analysis, pointing out the advantages and possible drawbacks connected to the widespread application of AI in the medical/mental health fields. The integration of mental health research and ML data science will facilitate the transition to personalized and effective medicine, and, to do so, it is important that researchers from psychological/ medical disciplines/health care professionals and data scientists all share a common background and vision of the current research.

Early-stage fusion of EEG and fNIRS improves classification of motor imagery

Introduction

Many research papers have reported successful implementation of hybrid brain-computer interfaces by complementarily combining EEG and fNIRS, to improve classification performance. However, modality or feature fusion of EEG and fNIRS was usually designed for specific user cases, which were generally customized and hard to be generalized. How to effectively utilize information from the two modalities was still unclear.

Methods

In this paper, we conducted a study to investigate the stage of bi-modal fusion based on EEG and fNIRS. A Y-shaped neural network was proposed and evaluated on an open dataset, which fuses the bimodal information in different stages.

Results

The results suggests that the early-stage fusion of EEG and fNIRS have significantly higher performance compared to middle-stage and late-stage fusion network configuration (N = 57, P < 0.05). With the proposed framework, the average accuracy of 29 participants reaches 76.21% in the left-or-right hand motor imagery task in leave-one-out cross-validation, using bi-modal data as network inputs respectively, which is in the same level as the state-of-the-art hybrid BCI methods based on EEG and fNIRS data.

Identifying Thematics in a Brain-Computer Interface Research

This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.

Functional near-infrared spectroscopy is a useful tool for multi-perspective psychobiological study of neurophysiological correlates of parenting behaviour

The quality of the relationship between caregiver and child has long-term effects on the cognitive and socio-emotional development of children. A process involved in human parenting is the bio-behavioural synchrony that occurs between the partners in the relationship during interaction. Through interaction, bio-behavioural synchronicity allows the adaptation of the physiological systems of the parent to those of the child and promotes the positive development and modelling of the child's social brain. The role of bio-behavioural synchrony in building social bonds could be investigated using functional near-infrared spectroscopy (fNIRS). In this paper we have (a) highlighted the importance of the quality of the caregiver-child relationship for the child's cognitive and socio-emotional development, as well as the relevance of infantile stimuli in the activation of parenting behaviour; (b) discussed the tools used in the study of the neurophysiological substrates of the parental response; (c) proposed fNIRS as a particularly suitable tool for the study of parental responses; and (d) underlined the need for a multi-systemic psychobiological approach to understand the mechanisms that regulate caregiver-child interactions and their bio-behavioural synchrony. We propose to adopt a multi-system psychobiological approach to the study of parental behaviour and social interaction.

Performance comparison of systemic activity correction in functional near-infrared spectroscopy for methods with and without short distance channels

Significance: Functional near-infrared spectroscopy (fNIRS) is a promising tool for neurofeedback (NFB) or brain-computer interfaces (BCIs). However, fNIRS signals are typically highly contaminated by systemic activity (SA) artifacts, and, if not properly corrected, NFB or BCIs run the risk of being based on noise instead of brain activity. This risk can likely be reduced by correcting for SA, in particular when short-distance channels (SDCs) are available. Literature comparing correction methods with and without SDCs is still sparse, specifically comparisons considering single trials are lacking. Aim: This study aimed at comparing the performance of SA correction methods with and without SDCs. Approach: Semisimulated and real motor task data of healthy older adults were used. Correction methods without SDCs included a simple and a more advanced spatial filter. Correction methods with SDCs included a regression approach considering only the closest SDC and two GLM-based methods, one including all eight SDCs and one using only two a priori selected SDCs as regressors. All methods were compared with data uncorrected for SA and correction performance was assessed with quality measures quantifying signal improvement and spatial specificity at single trial level. Results: All correction methods were found to improve signal quality and enhance spatial specificity as compared with the uncorrected data. Methods with SDCs usually outperformed methods without SDCs. Correction methods without SDCs tended to overcorrect the data. However, the exact pattern of results and the degree of differences observable between correction methods varied between semisimulated and real data, and also between quality measures. Conclusions: Overall, results confirmed that both Δ [ HbO ] and Δ [ HbR ] are affected by SA and that correction methods with SDCs outperform methods without SDCs. Nonetheless, improvements in signal quality can also be achieved without SDCs and should therefore be given priority over not correcting for SA.

Organic photodiodes: device engineering and applications

Organic photodiodes (OPDs) have shown great promise for potential applications in optical imaging, sensing, and communication due to their wide-range tunable photoelectrical properties, low-temperature facile processes, and excellent mechanical flexibility. Extensive research work has been carried out on exploring materials, device structures, physical mechanisms, and processing approaches to improve the performance of OPDs to the level of their inorganic counterparts. In addition, various system prototypes have been built based on the exhibited and attractive features of OPDs. It is vital to link the device optimal design and engineering to the system requirements and examine the existing deficiencies of OPDs towards practical applications, so this review starts from discussions on the required key performance metrics for different envisioned applications. Then the fundamentals of the OPD device structures and operation mechanisms are briefly introduced, and the latest development of OPDs for improving the key performance merits is reviewed. Finally, the trials of OPDs for various applications including wearable medical diagnostics, optical imagers, spectrometers, and light communications are reviewed, and both the promises and challenges are revealed.

fNIRS-based brain functional response to robot-assisted training for upper-limb in stroke patients with hemiplegia

Background

Robot-assisted therapy (RAT) has received considerable attention in stroke motor rehabilitation. Characteristics of brain functional response associated with RAT would provide a theoretical basis for choosing the appropriate protocol for a patient. However, the cortical response induced by RAT remains to be fully elucidated due to the lack of dynamic brain functional assessment tools.

Objective

To guide the implementation of clinical therapy, this study focused on the brain functional responses induced by RAT in patients with different degrees of motor impairment.

Methods

A total of 32 stroke patients were classified into a low score group (severe impairment, n = 16) and a high score group (moderate impairment, n = 16) according to the motor function of the upper limb and then underwent RAT training in assistive mode with simultaneous cerebral haemodynamic measurement by functional near-infrared spectroscopy (fNIRS). Functional connectivity (FC) and the hemisphere autonomy index (HAI) were calculated based on the wavelet phase coherence among fNIRS signals covering bilateral prefrontal, motor and occipital areas.

Results

Specific cortical network response related to RAT was observed in patients with unilateral moderate-to-severe motor deficits in the subacute stage. Compared with patients with moderate dysfunction, patients with severe impairment showed a wide range of significant FC responses in the bilateral hemispheres induced by RAT with the assistive mode, especially task-related involvement of ipsilesional supplementary motor areas.

Conclusion

Under assisted mode, RAT-related extensive cortical response in patients with severe dysfunction might contribute to brain functional organization during motor performance, which is considered the basic neural substrate of motor-related processes. In contrast, the limited cortical response related to RAT in patients with moderate dysfunction may indicate that the training intensity needs to be adjusted in time according to the brain functional state. fNIRS-based assessment of brain functional response assumes great importance for the customization of an appropriate protocol training in the clinical practice.

Home-based portable fNIRS-derived cortical laterality correlates with impairment and function in chronic stroke

Introduction

Improved understanding of the relationship between post-stroke rehabilitation interventions and functional motor outcomes could result in improvements in the efficacy of post-stroke physical rehabilitation. The laterality of motor cortex activity (M1-LAT) during paretic upper-extremity movement has been documented as a useful biomarker of post-stroke motor recovery. However, the expensive, labor intensive, and laboratory-based equipment required to take measurements of M1-LAT limit its potential clinical utility in improving post-stroke physical rehabilitation. The present study tested the ability of a mobile functional near-infrared spectroscopy (fNIRS) system (designed to enable independent measurement by stroke survivors) to measure cerebral hemodynamics at the motor cortex in the homes of chronic stroke survivors.

Methods

Eleven chronic stroke survivors, ranging widely in their level of upper-extremity motor deficit, used their stroke-affected upper-extremity to perform a simple unilateral movement protocol in their homes while a wireless prototype fNIRS headband took measurements at the motor cortex. Measures of participants' upper-extremity impairment and function were taken.

Results

Participants demonstrated either a typically lateralized response, with an increase in contralateral relative oxyhemoglobin (ΔHbO), or response showing a bilateral pattern of increase in ΔHbO during the motor task. During the simple unilateral task, M1-LAT correlated significantly with measures of both upper-extremity impairment and function, indicating that participants with more severe motor deficits had more a more atypical (i.e., bilateral) pattern of lateralization.

Discussion

These results indicate it is feasible to gain M1-LAT measures from stroke survivors in their homes using fNIRS. These findings represent a preliminary step toward the goals of using ergonomic functional neuroimaging to improve post-stroke rehabilitative care, via the capture of neural biomarkers of post-stroke motor recovery, and/or via use as part of an accessible rehabilitation brain-computer-interface.

Screening for Alzheimer's disease using prefrontal resting-state functional near-infrared spectroscopy

INTRODUCTION

Alzheimer's disease (AD) is neurodegenerative dementia that causes neurovascular dysfunction and cognitive impairment. Currently, 50 million people live with dementia worldwide, and there are nearly 10 million new cases every year. There is a need for relatively less costly and more objective methods of screening and early diagnosis.

METHODS

Functional near-infrared spectroscopy (fNIRS) systems are a promising solution for the early Detection of AD. For a practical clinically relevant system, a smaller number of optimally placed channels are clearly preferable. In this study, we investigated the number and locations of the best-performing fNIRS channels measuring prefrontal cortex activations. Twenty-one subjects diagnosed with AD and eighteen healthy controls were recruited for the study.

RESULTS

We have shown that resting-state fNIRS recordings from a small number of prefrontal locations provide a promising methodology for detecting AD and monitoring its progression. A high-density continuous-wave fNIRS system was first used to verify the relatively lower hemodynamic activity in the prefrontal cortical areas observed in patients with AD. By using the episode averaged standard deviation of the oxyhemoglobin concentration changes as features that were fed into a Support Vector Machine; we then showed that the accuracy of subsets of optical channels in predicting the presence and severity of AD was significantly above chance. The results suggest that AD can be detected with a 0.76 sensitivity score and a 0.68 specificity score while the severity of AD could be detected with a 0.75 sensitivity score and a 0.72 specificity score with ≤5 channels.

DISCUSSION

These scores suggest that fNIRS is a viable technology for conveniently detecting and monitoring AD as well as investigating underlying mechanisms of disease progression.

Decoding lexical tones and vowels in imagined tonal monosyllables using fNIRS signals

Objective.Speech is a common way of communication. Decoding verbal intent could provide a naturalistic communication way for people with severe motor disabilities. Active brain computer interaction (BCI) speller is one of the most commonly used speech BCIs. To reduce the spelling time of Chinese words, identifying vowels and tones that are embedded in imagined Chinese words is essential. Functional near-infrared spectroscopy (fNIRS) has been widely used in BCI because it is portable, non-invasive, safe, low cost, and has a relatively high spatial resolution.Approach.In this study, an active BCI speller based on fNIRS is presented by covertly rehearsing tonal monosyllables with vowels (i.e. /a/, /i/, /o/, and /u/) and four lexical tones in Mandarin Chinese (i.e. tones 1, 2, 3, and 4) for 10 s.Main results.fNIRS results showed significant differences in the right superior temporal gyrus between imagined vowels with tone 2/3/4 and those with tone 1 (i.e. more activations and stronger connections to other brain regions for imagined vowels with tones 2/3/4 than for those with tone 1). Speech-related areas for tone imagery (i.e. the right hemisphere) provided majority of information for identifying tones, while the left hemisphere had advantages in vowel identification. Having decoded both vowels and tones during the post-stimulus 15 s period, the average classification accuracies exceeded 40% and 70% in multiclass (i.e. four classes) and binary settings, respectively. To spell words more quickly, the time window size for decoding was reduced from 15 s to 2.5 s while the classification accuracies were not significantly reduced.Significance.For the first time, this work demonstrated the possibility of discriminating lexical tones and vowels in imagined tonal syllables simultaneously. In addition, the reduced time window for decoding indicated that the spelling time of Chinese words could be significantly reduced in the fNIRS-based BCIs.

Resting-state functional connectivity for determining outcomes in upper extremity function after stroke: A functional near-infrared spectroscopy study

OBJECTIVE

Functional near-infrared spectroscopy (fNIRS) is a non-invasive and promising tool to map the brain functional networks in stroke recovery. Our study mainly aimed to use fNIRS to detect the different patterns of resting-state functional connectivity (RSFC) in subacute stroke patients with different degrees of upper extremity motor impairment defined by Fugl-Meyer motor assessment of upper extremity (FMA-UE). The second aim was to investigate the association between FMA-UE scores and fNIRS-RSFC among different regions of interest (ROIs) in stroke patients.

METHODS

Forty-nine subacute (2 weeks-6 months) stroke patients with subcortical lesions were enrolled and were classified into three groups based on FMA-UE scores: mild impairment (n = 17), moderate impairment (n = 13), and severe impairment (n = 19). All patients received FMA-UE assessment and 10-min resting-state fNIRS monitoring. The fNIRS signals were recorded over seven ROIs: bilateral dorsolateral prefrontal cortex (DLPFC), middle prefrontal cortex (MPFC), bilateral primary motor cortex (M1), and bilateral primary somatosensory cortex (S1). Functional connectivity (FC) was calculated by correlation coefficients between each channel and each ROI pair. To reveal the comprehensive differences in FC among three groups, we compared FC on the group level and ROI level. In addition, to determine the associations between FMA-UE scores and RSFC among different ROIs, Spearman's correlation analyses were performed with a significance threshold of p < 0.05. For easy comparison, we defined the left hemisphere as the ipsilesional hemisphere and flipped the lesional right hemisphere in MATLAB R2013b.

RESULTS

For the group-level comparison, the one-way ANOVA and post-hoc t-tests (mild vs. moderate; mild vs. severe; moderate vs. severe) showed that there was a significant difference among three groups (F = 3.42, p = 0.04) and the group-averaged FC in the mild group (0.64 ± 0.14) was significantly higher than that in the severe group (0.53 ± 0.14, p = 0.013). However, there were no significant differences between the mild and moderate group (MD ± SE = 0.05 ± 0.05, p = 0.35) and between the moderate and severe group (MD ± SE = 0.07 ± 0.05, p = 0.16). For the ROI-level comparison, the severe group had significantly lower FC of ipsilesional DLPFC-ipsilesional M1 [p = 0.015, false discovery rate (FDR)-corrected] and ipsilesional DLPFC-contralesional M1 (p = 0.035, FDR-corrected) than those in the mild group. Moreover, the result of Spearman's correlation analyses showed that there were significant correlations between FMA-UE scores and FC of the ipsilesional DLPFC-ipsilesional M1 (r = 0.430, p = 0.002), ipsilesional DLPFC-contralesional M1 (r = 0.388, p = 0.006), ipsilesional DLPFC-MPFC (r = 0.365, p = 0.01), and ipsilesional DLPFC-contralesional DLPFC (r = 0.330, p = 0.021).

CONCLUSION

Our findings indicate that different degrees of post-stroke upper extremity impairment reflect different RSFC patterns, mainly in the connection between DLPFC and bilateral M1. The association between FMA-UE scores and the FC of ipsilesional DLPFC-associated ROIs suggests that the ipsilesional DLPFC may play an important role in motor-related plasticity. These findings can help us better understand the neurophysiological mechanisms of upper extremity motor impairment and recovery in subacute stroke patients from different perspectives. Furthermore, it sheds light on the ipsilesional DLPFC-bilateral M1 as a possible neuromodulation target.

Effects of different exercise intensities of race-walking on brain functional connectivity as assessed by functional near-infrared spectroscopy

Introduction

Race-walking is a sport that mimics normal walking and running. Previous studies on sports science mainly focused on the cardiovascular and musculoskeletal systems. However, there is still a lack of research on the central nervous system, especially the real-time changes in brain network characteristics during race-walking exercise. This study aimed to use a network perspective to investigate the effects of different exercise intensities on brain functional connectivity.

Materials and methods

A total of 16 right-handed healthy young athletes were recruited as participants in this study. The cerebral cortex concentration of oxyhemoglobin was measured by functional near-infrared spectroscopy in the bilateral prefrontal cortex (PFC), the motor cortex (MC) and occipital cortex (OC) during resting and race-walking states. Three specific periods as time windows corresponding to different exercise intensities were divided from the race-walking time of participants, including initial, intermediate and sprint stages. The brain activation and functional connectivity (FC) were calculated to describe the 0.01-0.1 Hz frequency-specific cortical activities.

Results

Compared to the resting state, FC changes mainly exist between MC and OC in the initial stage, while PFC was involved in FC changes in the intermediate stage, and FC changes in the sprint stage were widely present in PFC, MC and OC. In addition, from the initial-development to the sprint stage, the significant changes in FC were displayed in PFC and MC.

Conclusion

This brain functional connectivity-based study confirmed that hemodynamic changes at different exercise intensities reflected different brain network-specific characteristics. During race-walking exercise, more extensive brain activation might increase information processing speed. Increased exercise intensity could facilitate the integration of neural signals such as proprioception, motor control and motor planning, which may be an important factor for athletes to maintain sustained motor coordination and activity control at high intensity. This study was beneficial to understanding the neural mechanisms of brain networks under different exercise intensities.

Effects of Personalized Cognitive Training with the Machine Learning Algorithm on Neural Efficiency in Healthy Younger Adults

To date, neural efficiency, an ability to economically utilize mental resources, has not been investigated after cognitive training. The purpose of this study was to provide customized cognitive training and confirm its effect on neural efficiency by investigating prefrontal cortex (PFC) activity using functional near-infrared spectroscopy (fNIRS). Before training, a prediction algorithm based on the PFC activity with logistic regression was used to predict the customized difficulty level with 86% accuracy by collecting data when subjects performed four kinds of cognitive tasks. In the next step, the intervention study was designed using one pre-posttest group. Thirteen healthy adults participated in the virtual reality (VR)-based spatial cognitive training, which was conducted four times a week for 30 min for three weeks with customized difficulty levels for each session. To measure its effect, the trail-making test (TMT) and hemodynamic responses were measured for executive function and PFC activity. During the training, VR-based spatial cognitive performance was improved, and hemodynamic values were gradually increased as the training sessions progressed. In addition, after the training, the performance on the trail-making task (TMT) demonstrated a statistically significant improvement, and there was a statistically significant decrease in the PFC activity. The improved performance on the TMT coupled with the decreased PFC activity could be regarded as training-induced neural efficiency. These results suggested that personalized cognitive training could be effective in improving executive function and neural efficiency.

Modulating swallowing-related functional connectivity and behavior via modified pharyngeal electrical stimulation: A functional near-infrared spectroscopy evidence

Introduction

Modified pharyngeal electrical stimulation (mPES) is a novel therapeutic modality for patients with neurogenic dysphagia. However, the underlying neural mechanism remains poorly understood. This study aimed to use functional near-infrared spectroscopy (fNIRS) to explore the influence of mPES on swallowing-related frequency-specific neural networks and ethology.

Methods

Twenty-two healthy right-handed volunteers participated in the study. Each participant was randomly assigned to either the sham or the mPES group and provided a 10-min intervention program every day for 5 days. Oxyhemoglobin and deoxyhemoglobin concentration changes verified by fNIRS were recorded on days 1, 3, and 5. Five characteristic frequency signals (0.0095-2 Hz) were identified using the wavelet transform method. To calculate frequency-specific functional connectivity, wavelet phase coherence (WPCO) was adopted. Furthermore, behavioral performance was assessed pre- and post-mPES using a 150 ml-water swallowing stress test.

Results

Compared with sham stimulation on day 1, the significantly decreased WPCO values were mainly associated with the dorsolateral prefrontal lobe, Broca's area, and middle temporal lobe. Compared with the sham mPES on day 1, the mPES showed a noticeable effect on the total swallow duration. Compared with the baseline, the WPCO values on days 3 and 5 showed a stepwise decrease in connectivity with the application of mPES. Furthermore, the decreased WPCO was associated with a shortened time per swallow after mPES.

Conclusions

The mPES could modulate swallowing-related frequency-specific neural networks and evoke swallowing cortical processing more efficiently. This was associated with improved performance in a water swallowing stress test in healthy participants.

Vibrotactile enhancement in hand rehabilitation has a reinforcing effect on sensorimotor brain activities

Objective

Stroke patients often suffer from hand dysfunction or loss of tactile perception, which in turn interferes with hand rehabilitation. Tactile-enhanced multi-sensory feedback rehabilitation is an approach worth considering, but its effectiveness has not been well studied. By using functional near-infrared spectroscopy (fNIRS) to analyze the causal activity patterns in the sensorimotor cortex, the present study aims to investigate the cortical hemodynamic effects of hand rehabilitation training when tactile stimulation is applied, and to provide a basis for rehabilitation program development.

Methods

A vibrotactile enhanced pneumatically actuated hand rehabilitation device was tested on the less-preferred hand of 14 healthy right-handed subjects. The training tasks consisted of move hand and observe video (MO), move hand and vibration stimulation (MV), move hand, observe video, and vibration stimulation (MOV), and a contrast resting task. Region of interest (ROI), a laterality index (LI), and causal brain network analysis methods were used to explore the brain’s cortical blood flow response to a multi-sensory feedback rehabilitation task from multiple perspectives.

Results

(1) A more pronounced contralateral activation in the right-brain region occurred under the MOV stimulation. Rehabilitation tasks containing vibrotactile enhancement (MV and MOV) had significantly more oxyhemoglobin than the MO task at 5 s after the task starts, indicating faster contralateral activation in sensorimotor brain regions. (2) Five significant lateralized channel connections were generated under the MV and MOV tasks (p < 0.05), one significant lateralized channel connection was generated by the MO task, and the Rest were not, showing that MV and MOV caused stronger lateralization activation. (3) We investigated all thresholds of granger causality (GC) resulting in consistent relative numbers of effect connections. MV elicited stronger causal interactions between the left and right cerebral hemispheres, and at the GC threshold of 0.4, there were 13 causal network connection pairs for MV, 7 for MO, and 9 for MOV.

Conclusion

Vibrotactile cutaneous stimulation as a tactile enhancement can produce a stronger stimulation of the brain’s sensorimotor brain areas, promoting the establishment of neural pathways, and causing a richer effect between the left and right cerebral hemispheres. The combination of kinesthetic, vibrotactile, and visual stimulation can achieve a more prominent training efficiency from the perspective of functional cerebral hemodynamics.

Motion artifacts removal and evaluation techniques for functional near-infrared spectroscopy signals: A review

With the emergence of an increasing number of functional near-infrared spectroscopy (fNIRS) devices, the significant deterioration in measurement caused by motion artifacts has become an essential research topic for fNIRS applications. However, a high requirement for mathematics and programming limits the number of related researches. Therefore, here we provide the first comprehensive review for motion artifact removal in fNIRS aiming to (i) summarize the latest achievements, (ii) present the significant solutions and evaluation metrics from the perspective of application and reproduction, and (iii) predict future topics in the field. The present review synthesizes information from fifty-one journal articles (screened according to three criteria). Three hardware-based solutions and nine algorithmic solutions are summarized, and their application requirements (compatible signal types, the availability for online applications, and limitations) and extensions are discussed. Five metrics for noise suppression and two metrics for signal distortion were synthesized to evaluate the motion artifact removal methods. Moreover, we highlight three deficiencies in the existing research: (i) The balance between the use of auxiliary hardware and that of an algorithmic solution is not clarified; (ii) few studies mention the filtering delay of the solutions, and (iii) the robustness and stability of the solution under extreme application conditions are not discussed.

A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI

Decoding neural responses from multimodal information sources, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), has the transformative potential to advance hybrid brain-computer interfaces (hBCIs). However, existing modest performance improvement of hBCIs might be attributed to the lack of computational frameworks that exploit complementary synergistic properties in multimodal features. This study proposes a multimodal data fusion framework to represent and decode synergistic multimodal motor imagery (MI) neural responses. We hypothesize that exploiting EEG nonlinear dynamics adds a new informative dimension to the commonly combined EEG-fNIRS features and will ultimately increase the synergy between EEG and fNIRS features toward an enhanced hBCI. The EEG nonlinear dynamics were quantified by extracting graph-based recurrence quantification analysis (RQA) features to complement the commonly used spectral features for an enhanced multimodal configuration when combined with fNIRS. The high-dimensional multimodal features were further given to a feature selection algorithm relying on the least absolute shrinkage and selection operator (LASSO) for fused feature selection. Linear support vector machine (SVM) was then used to evaluate the framework. The mean hybrid classification performance improved by up to 15% and 4% compared to the unimodal EEG and fNIRS, respectively. The proposed graph-based framework substantially increased the contribution of EEG features for hBCI classification from 28.16% up to 52.9% when introduced the nonlinear dynamics and improved the performance by approximately 2%. These findings suggest that graph-based nonlinear dynamics can increase the synergy between EEG and fNIRS features for an enhanced MI response representation that is not dominated by a single modality.