fNIRS-based brain-computer interfaces: a review

LASSO Homotopy-Based Sparse Representation Classification for fNIRS-BCI

Brain-computer interface (BCI) systems based on functional near-infrared spectroscopy (fNIRS) have been used as a way of facilitating communication between the brain and peripheral devices. The BCI provides an option to improve the walking pattern of people with poor walking dysfunction, by applying a rehabilitation process. A state-of-the-art step-wise BCI system includes data acquisition, pre-processing, channel selection, feature extraction, and classification. In fNIRS-based BCI (fNIRS-BCI), channel selection plays a vital role in enhancing the classification accuracy of the BCI problem. In this study, the concentration of blood oxygenation (HbO) in a resting state and in a walking state was used to decode the walking activity and the resting state of the subject, using channel selection by Least Absolute Shrinkage and Selection Operator (LASSO) homotopy-based sparse representation classification. The fNIRS signals of nine subjects were collected from the left hemisphere of the primary motor cortex. The subjects performed the task of walking on a treadmill for 10 s, followed by a 20 s rest. Appropriate filters were applied to the collected signals to remove motion artifacts and physiological noises. LASSO homotopy-based sparse representation was used to select the most significant channels, and then classification was performed to identify walking and resting states. For comparison, the statistical spatial features of mean, peak, variance, and skewness, and their combination, were used for classification. The classification results after channel selection were then compared with the classification based on the extracted features. The classifiers used for both methods were linear discrimination analysis (LDA), support vector machine (SVM), and logistic regression (LR). The study found that LASSO homotopy-based sparse representation classification successfully discriminated between the walking and resting states, with a better average classification accuracy (p < 0.016) of 91.32%. This research provides a step forward in improving the classification accuracy of fNIRS-BCI systems. The proposed methodology may also be used for rehabilitation purposes, such as controlling wheelchairs and prostheses, as well as an active rehabilitation training technique for patients with motor dysfunction.

Involvement of the Rostromedial Prefrontal Cortex in Human-Robot Interaction: fNIRS Evidence From a Robot-Assisted Motor Task

Assistive exoskeleton robots are being widely applied in neurorehabilitation to improve upper-limb motor and somatosensory functions. During robot-assisted exercises, the central nervous system appears to highly attend to external information-processing (IP) to efficiently interact with robotic assistance. However, the neural mechanisms underlying this process remain unclear. The rostromedial prefrontal cortex (rmPFC) may be the core of the executive resource allocation that generates biases in the allocation of processing resources toward an external IP according to current behavioral demands. Here, we used functional near-infrared spectroscopy to investigate the cortical activation associated with executive resource allocation during a robot-assisted motor task. During data acquisition, participants performed a right-arm motor task using elbow flexion-extension movements in three different loading conditions: robotic assistive loading (ROB), resistive loading (RES), and non-loading (NON). Participants were asked to strive for kinematic consistency in their movements. A one-way repeated measures analysis of variance and general linear model-based methods were employed to examine task-related activity. We demonstrated that hemodynamic responses in the ventral and dorsal rmPFC were higher during ROB than during NON. Moreover, greater hemodynamic responses in the ventral rmPFC were observed during ROB than during RES. Increased activation in ventral and dorsal rmPFC subregions may be involved in the executive resource allocation that prioritizes external IP during human-robot interactions. In conclusion, these findings provide novel insights regarding the involvement of executive control during a robot-assisted motor task.

fMRI-based validation of continuous-wave fNIRS of supplementary motor area activation during motor execution and motor imagery

Compared to functional magnetic resonance imaging (fMRI), functional near infrared spectroscopy (fNIRS) has several advantages that make it particularly interesting for neurofeedback (NFB). A pre-requisite for NFB applications is that with fNIRS, signals from the brain region of interest can be measured. This study focused on the supplementary motor area (SMA). Healthy older participants (N = 16) completed separate continuous-wave (CW-) fNIRS and (f)MRI sessions. Data were collected for executed and imagined hand movements (motor imagery, MI), and for MI of whole body movements. Individual anatomical data were used to (i) define the regions of interest for fMRI analysis, to (ii) extract the fMRI BOLD response from the cortical regions corresponding to the fNIRS channels, and (iii) to select fNIRS channels. Concentration changes in oxygenated ([Formula: see text]) and deoxygenated ([Formula: see text]) hemoglobin were considered in the analyses. Results revealed subtle differences between the different MI tasks, indicating that for whole body MI movements as well as for MI of hand movements [Formula: see text] is the more specific signal. Selection of the fNIRS channel set based on individual anatomy did not improve the results. Overall, the study indicates that in terms of spatial specificity and task sensitivity SMA activation can be reliably measured with CW-fNIRS.

Analyzing Classification Performance of fNIRS-BCI for Gait Rehabilitation Using Deep Neural Networks

This research presents a brain-computer interface (BCI) framework for brain signal classification using deep learning (DL) and machine learning (ML) approaches on functional near-infrared spectroscopy (fNIRS) signals. fNIRS signals of motor execution for walking and rest tasks are acquired from the primary motor cortex in the brain's left hemisphere for nine subjects. DL algorithms, including convolutional neural networks (CNNs), long short-term memory (LSTM), and bidirectional LSTM (Bi-LSTM) are used to achieve average classification accuracies of 88.50%, 84.24%, and 85.13%, respectively. For comparison purposes, three conventional ML algorithms, support vector machine (SVM), k-nearest neighbor (k-NN), and linear discriminant analysis (LDA) are also used for classification, resulting in average classification accuracies of 73.91%, 74.24%, and 65.85%, respectively. This study successfully demonstrates that the enhanced performance of fNIRS-BCI can be achieved in terms of classification accuracy using DL approaches compared to conventional ML approaches. Furthermore, the control commands generated by these classifiers can be used to initiate and stop the gait cycle of the lower limb exoskeleton for gait rehabilitation.

Spatial complexity method for tracking brain development and degeneration using functional near-infrared spectroscopy

Brain complexity analysis using functional near-infrared spectroscopy (fNIRS) has attracted attention as a biomarker for evaluating brain development and degeneration processes. However, most methods have focused on the temporal scale without capturing the spatial complexity. In this study, we propose a spatial time-delay entropy (STDE) method as the spatial complexity measure based on the time-delay measure between two oxy-hemoglobin (Δ[HbO]) or two deoxy-hemoglobin (Δ[Hb]) oscillations within the 0.01-0.1 Hz frequency band. To do this, we analyze fNIRS signals recorded from infants in their sleeping state, children, adults, and healthy seniors in their resting states. We also evaluate the effects of various noise to STDE calculations and STDE's performance in distinguishing various developmental age groups. Lastly, we compare the results with the normalized global spatial complexity (NGSC) and sample entropy (SampEn) measures. Among these measures, STDE_HbO (STDE based on Δ[HbO] oscillations) performs best. The STDE value increases with age throughout childhood (p < 0.001), and then decreases in adults and healthy seniors in the 0.01-0.1 Hz frequency band. This trajectory correlates with cerebrovascular development and degeneration. These findings demonstrate that STDE can be used as a new tool for tracking cerebrovascular development and degeneration across a lifespan based on the fNIRS resting-state measurements.

Intention Detection Strategies for Robotic Upper-Limb Orthoses: A Scoping Review Considering Usability, Daily Life Application, and User Evaluation

Wearable robotic upper limb orthoses (ULO) are promising tools to assist or enhance the upper-limb function of their users. While the functionality of these devices has continuously increased, the robust and reliable detection of the user's intention to control the available degrees of freedom remains a major challenge and a barrier for acceptance. As the information interface between device and user, the intention detection strategy (IDS) has a crucial impact on the usability of the overall device. Yet, this aspect and the impact it has on the device usability is only rarely evaluated with respect to the context of use of ULO. A scoping literature review was conducted to identify non-invasive IDS applied to ULO that have been evaluated with human participants, with a specific focus on evaluation methods and findings related to functionality and usability and their appropriateness for specific contexts of use in daily life. A total of 93 studies were identified, describing 29 different IDS that are summarized and classified according to a four-level classification scheme. The predominant user input signal associated with the described IDS was electromyography (35.6%), followed by manual triggers such as buttons, touchscreens or joysticks (16.7%), as well as isometric force generated by residual movement in upper-limb segments (15.1%). We identify and discuss the strengths and weaknesses of IDS with respect to specific contexts of use and highlight a trade-off between performance and complexity in selecting an optimal IDS. Investigating evaluation practices to study the usability of IDS, the included studies revealed that, primarily, objective and quantitative usability attributes related to effectiveness or efficiency were assessed. Further, it underlined the lack of a systematic way to determine whether the usability of an IDS is sufficiently high to be appropriate for use in daily life applications. This work highlights the importance of a user- and application-specific selection and evaluation of non-invasive IDS for ULO. For technology developers in the field, it further provides recommendations on the selection process of IDS as well as to the design of corresponding evaluation protocols.

Emotion Recognition Based on Dynamic Energy Features Using a Bi-LSTM Network

Among electroencephalogram (EEG) signal emotion recognition methods based on deep learning, most methods have difficulty in using a high-quality model due to the low resolution and the small sample size of EEG images. To solve this problem, this study proposes a deep network model based on dynamic energy features. In this method, first, to reduce the noise superposition caused by feature analysis and extraction, the concept of an energy sequence is proposed. Second, to obtain the feature set reflecting the time persistence and multicomponent complexity of EEG signals, the construction method of the dynamic energy feature set is given. Finally, to make the network model suitable for small datasets, we used fully connected layers and bidirectional long short-term memory (Bi-LSTM) networks. To verify the effectiveness of the proposed method, we used leave one subject out (LOSO) and 10-fold cross validation (CV) strategies to carry out experiments on the SEED and DEAP datasets. The experimental results show that the accuracy of the proposed method can reach 89.42% (SEED) and 77.34% (DEAP).

Multi-Modal Integration of EEG-fNIRS for Characterization of Brain Activity Evoked by Preferred Music

Music can effectively improve people's emotions, and has now become an effective auxiliary treatment method in modern medicine. With the rapid development of neuroimaging, the relationship between music and brain function has attracted much attention. In this study, we proposed an integrated framework of multi-modal electroencephalogram (EEG) and functional near infrared spectroscopy (fNIRS) from data collection to data analysis to explore the effects of music (especially personal preferred music) on brain activity. During the experiment, each subject was listening to two different kinds of music, namely personal preferred music and neutral music. In analyzing the synchronization signals of EEG and fNIRS, we found that music promotes the activity of the brain (especially the prefrontal lobe), and the activation induced by preferred music is stronger than that of neutral music. For the multi-modal features of EEG and fNIRS, we proposed an improved Normalized-ReliefF method to fuse and optimize them and found that it can effectively improve the accuracy of distinguishing between the brain activity evoked by preferred music and neutral music (up to 98.38%). Our work provides an objective reference based on neuroimaging for the research and application of personalized music therapy.

Detecting Fear of Heights Response to a Virtual Reality Environment Using Functional Near-Infrared Spectroscopy

To enable virtual reality exposure therapy (VRET) that treats anxiety disorders by gradually exposing the patient to fear using virtual reality (VR), it is important to monitor the patient's fear levels during the exposure. Despite the evidence of a fear circuit in the brain as reflected by functional near-infrared spectroscopy (fNIRS), the measurement of fear response in highly immersive VR using fNIRS is limited, especially in combination with a head-mounted display (HMD). In particular, it is unclear to what extent fNIRS can differentiate users with and without anxiety disorders and detect fear response in a highly ecological setting using an HMD. In this study, we investigated fNIRS signals captured from participants with and without a fear of height response. To examine the extent to which fNIRS signals of both groups differ, we conducted an experiment during which participants with moderate fear of heights and participants without it were exposed to VR scenarios involving heights and no heights. The between-group statistical analysis shows that the fNIRS data of the control group and the experimental group are significantly different only in the channel located close to right frontotemporal lobe, where the grand average oxygenated hemoglobin Δ[HbO] contrast signal of the experimental group exceeds that of the control group. The within-group statistical analysis shows significant differences between the grand average Δ[HbO] contrast values during fear responses and those during no-fear responses, where the Δ[HbO] contrast values of the fear responses were significantly higher than those of the no-fear responses in the channels located towards the frontal part of the prefrontal cortex. Also, the channel located close to frontocentral lobe was found to show significant difference for the grand average deoxygenated hemoglobin contrast signals. Support vector machine-based classifier could detect fear responses at an accuracy up to 70% and 74% in subject-dependent and subject-independent classifications, respectively. The results demonstrate that cortical hemodynamic responses of a control group and an experimental group are different to a considerable extent, exhibiting the feasibility and ecological validity of the combination of VR-HMD and fNIRS to elicit and detect fear responses. This research thus paves a way toward the a brain-computer interface to effectively manipulate and control VRET.

Immediate acupuncture with GB34 for biliary colic: protocol for a randomised controlled neuroimaging trial

INTRODUCTION

As the main manifestation of gallstone disease, biliary colic (BC) is an episodic attack that brings patients severe pain in the right upper abdominal quadrant. Although acupuncture has been documented with significance to lead to pain relief, the immediate analgesia of acupuncture for BC still needs to be verified, and the underlying mechanism has yet to be covered. Therefore, this trial aims first to verify the immediate pain-alleviation characteristic of acupuncture for BC, then to explore its influence on the peripheral sensitised acupoint and central brain activity.

METHODS AND ANALYSIS

This is a randomised controlled, paralleled clinical trial, with patients and outcome assessors blinded. Seventy-two patients with gallbladder stone disease presenting with BC will be randomised into a verum acupuncture group and the sham acupuncture group. Both groups will receive one session of immediate acupuncture treatment. Improvements in patients' BC will be evaluated by the Numeric Rating Scale, and the pain threshold of acupoints will also be detected before and after treatment. During treatment, brain neural activity will be monitored with functional near-infrared spectroscopy (fNIRS), and the needle sensation will be rated. Clinical and fNIRS data will be analysed, respectively, to validate the acupuncture effect, and correlation analysis will be conducted to investigate the relationship between pain relief and peripheral-cerebral functional changes.

ETHICS AND DISSEMINATION

This trial has been approved by the institutional review boards and ethics committees of the First Teaching Hospital of Chengdu University of Traditional Chinese Medicine, with the ethical approval identifier 2019 KL-029, and the institutional review boards and ethics committees of the First People's Hospital of Longquanyi District, with the ethical approval identifier AF-KY-2020071. The results of this trial will be disseminated through peer-reviewed publications and conference abstracts or posters.

TRIAL REGISTRATION NUMBER

CTR2000034432.

Are Brain-Computer Interfaces Feasible With Integrated Photonic Chips?

The present paper examines the viability of a radically novel idea for brain-computer interface (BCI), which could lead to novel technological, experimental, and clinical applications. BCIs are computer-based systems that enable either one-way or two-way communication between a living brain and an external machine. BCIs read-out brain signals and transduce them into task commands, which are performed by a machine. In closed loop, the machine can stimulate the brain with appropriate signals. In recent years, it has been shown that there is some ultraweak light emission from neurons within or close to the visible and near-infrared parts of the optical spectrum. Such ultraweak photon emission (UPE) reflects the cellular (and body) oxidative status, and compelling pieces of evidence are beginning to emerge that UPE may well play an informational role in neuronal functions. In fact, several experiments point to a direct correlation between UPE intensity and neural activity, oxidative reactions, EEG activity, cerebral blood flow, cerebral energy metabolism, and release of glutamate. Therefore, we propose a novel skull implant BCI that uses UPE. We suggest that a photonic integrated chip installed on the interior surface of the skull may enable a new form of extraction of the relevant features from the UPE signals. In the current technology landscape, photonic technologies are advancing rapidly and poised to overtake many electrical technologies, due to their unique advantages, such as miniaturization, high speed, low thermal effects, and large integration capacity that allow for high yield, volume manufacturing, and lower cost. For our proposed BCI, we are making some very major conjectures, which need to be experimentally verified, and therefore we discuss the controversial parts, feasibility of technology and limitations, and potential impact of this envisaged technology if successfully implemented in the future.

Transformer Model for Functional Near-Infrared Spectroscopy Classification

Functional near-infrared spectroscopy (fNIRS) is a promising neuroimaging technology. The fNIRS classification problem has always been the focus of the brain-computer interface (BCI). Inspired by the success of Transformer based on self-attention mechanism in the fields of natural language processing and computer vision, we propose an fNIRS classification network based on Transformer, named fNIRS-T. We explore the spatial-level and channel-level representation of fNIRS signals to improve data utilization and network representation capacity. Besides, a preprocessing module, which consists of one-dimensional average pooling and layer normalization, is designed to replace filtering and baseline correction of data preprocessing. It makes fNIRS-T an end-to-end network, called fNIRS-PreT. Compared with traditional machine learning classifiers, convolutional neural network (CNN), and long short-term memory (LSTM), the proposed models obtain the best accuracy on three open-access datasets. Specifically, in the most extensive ternary classification task (30 subjects) that includes three types of overt movements, fNIRS-T, CNN, and LSTM obtain 75.49%, 72.89%, and 61.94% on test sets, respectively. Compared to traditional classifiers, fNIRS-T is at least 27.41% higher than statistical features and 6.79% higher than well-designed features. In the individual subject experiment of the ternary classification task, fNIRS-T achieves an average subject accuracy of 78.22% and surpasses CNN and LSTM by a large margin of +4.75% and +11.33%. fNIRS-PreT using raw data also achieves competitive performance to fNIRS-T. Therefore, the proposed models improve the performance of fNIRS-based BCI significantly.

Investigating the cognitive capacity constraints of an ICU care team using a systems engineering approach

Background

ICU operational conditions may contribute to cognitive overload and negatively impact on clinical decision making. We aimed to develop a quantitative model to investigate the association between the operational conditions and the quantity of medication orders as a measurable indicator of the multidisciplinary care team’s cognitive capacity.

Methods

The temporal data of patients at one medical ICU (MICU) of Mayo Clinic in Rochester, MN between February 2016 to March 2018 was used. This dataset includes a total of 4822 unique patients admitted to the MICU and a total of 6240 MICU admissions. Guided by the Systems Engineering Initiative for Patient Safety model, quantifiable measures attainable from electronic medical records were identified and a conceptual framework of distributed cognition in ICU was developed. Univariate piecewise Poisson regression models were built to investigate the relationship between system-level workload indicators, including patient census and patient characteristics (severity of illness, new admission, and mortality risk) and the quantity of medication orders, as the output of the care team’s decision making.

Results

Comparing the coefficients of different line segments obtained from the regression models using a generalized F-test, we identified that, when the ICU was more than 50% occupied (patient census > 18), the number of medication orders per patient per hour was significantly reduced (average = 0.74; standard deviation (SD) = 0.56 vs. average = 0.65; SD = 0.48; p < 0.001). The reduction was more pronounced (average = 0.81; SD = 0.59 vs. average = 0.63; SD = 0.47; p < 0.001), and the breakpoint shifted to a lower patient census (16 patients) when at a higher presence of severely-ill patients requiring invasive mechanical ventilation during their stay, which might be encountered in an ICU treating patients with COVID-19.

Conclusions

Our model suggests that ICU operational factors, such as admission rates and patient severity of illness may impact the critical care team’s cognitive function and result in changes in the production of medication orders. The results of this analysis heighten the importance of increasing situational awareness of the care team to detect and react to changing circumstances in the ICU that may contribute to cognitive overload.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12871-021-01548-7.

Characterizing reproducibility of cerebral hemodynamic responses when applying short-channel regression in functional near-infrared spectroscopy

Significance: Functional near-infrared spectroscopy (fNIRS) enables the measurement of brain activity noninvasively. Optical neuroimaging with fNIRS has been shown to be reproducible on the group level and hence is an excellent research tool, but the reproducibility on the single-subject level is still insufficient, challenging the use for clinical applications. Aim: We investigated the effect of short-channel regression (SCR) as an approach to obtain fNIRS measurements with higher reproducibility on a single-subject level. SCR simultaneously considers contributions from long- and short-separation channels and removes confounding physiological changes through the regression of the short-separation channel information. Approach: We performed a test-retest study with a hand grasping task in 15 healthy subjects using a wearable fNIRS device, optoHIVE. Relevant brain regions were localized with transcranial magnetic stimulation to ensure correct placement of the optodes. Reproducibility was assessed by intraclass correlation, correlation analysis, mixed effects modeling, and classification accuracy of the hand grasping task. Further, we characterized the influence of SCR on reproducibility. Results: We found a high reproducibility of fNIRS measurements on a single-subject level (ICC single = 0.81 and correlation r = 0.81 ). SCR increased the reproducibility from 0.64 to 0.81 (ICC single ) but did not affect classification (85% overall accuracy). Significant intersubject variability in the reproducibility was observed and was explained by Mayer wave oscillations and low raw signal strength. The raw signal-to-noise ratio (threshold at 40 dB) allowed for distinguishing between persons with weak and strong activations. Conclusions: We report, for the first time, that fNIRS measurements are reproducible on a single-subject level using our optoHIVE fNIRS system and that SCR improves reproducibility. In addition, we give a benchmark to easily assess the ability of a subject to elicit sufficiently strong hemodynamic responses. With these insights, we pave the way for the reliable use of fNIRS neuroimaging in single subjects for neuroscientific research and clinical applications.

MOCA: a systematic toolbox for designing and assessing modular functional near-infrared brain imaging probes

Significance

The expansion of functional near-infrared spectroscopy (fNIRS) systems toward broader utilities has led to the emergence of modular fNIRS systems composed of repeating optical source/detector modules. Compared to conventional fNIRS systems, modular fNIRS systems are more compact and flexible, making wearable and long-term monitoring possible. However, the large number of design parameters makes understanding their impact on a probe's performance a daunting task.

Aim

We aim to create a systematic software platform to facilitate the design, characterization, and comparison of modular fNIRS probes.

Approach

Our software-modular optode configuration analyzer (MOCA)-implements semi-automatic algorithms that assist in tessellating user-specified regions-of-interest, in interconnecting modules of various shapes, and in quantitatively comparing probe performance using metrics, such as spatial channel distributions and average brain sensitivity of the resulting probes. There is also support for limited parameter sweeping capabilities.

Results

Through several examples, we show that users can use MOCA to design and optimize modular fNIRS probes, study trade-offs between several module shapes, improve brain sensitivity in probes via module re-orientation, and enhance probe performance via adjusting module spatial layouts.

Conclusion

Despite its simplicity, our modular probe design platform offers a framework to describe and quantitatively assess probes made by modules, opening a new door for the growing fNIRS user community to approach the challenging problem of module- and probe-parameter selection and fine-tuning.

Classification of Game Demand and the Presence of Experimental Pain Using Functional Near-Infrared Spectroscopy

Pain tolerance can be increased by the introduction of an active distraction, such as a computer game. This effect has been found to be moderated by game demand, i.e., increased game demand = higher pain tolerance. A study was performed to classify the level of game demand and the presence of pain using implicit measures from functional Near-InfraRed Spectroscopy (fNIRS) and heart rate features from an electrocardiogram (ECG). Twenty participants played a racing game that was configured to induce low (Easy) or high (Hard) levels of demand. Both Easy and Hard levels of game demand were played with or without the presence of experimental pain using the cold pressor test protocol. Eight channels of fNIRS data were recorded from a montage of frontal and central-parietal sites located on the midline. Features were generated from these data, a subset of which were selected for classification using the RELIEFF method. Classifiers for game demand (Easy vs. Hard) and pain (pain vs. no-pain) were developed using five methods: Support Vector Machine (SVM), k-Nearest Neighbour (kNN), Naive Bayes (NB) and Random Forest (RF). These models were validated using a ten fold cross-validation procedure. The SVM approach using features derived from fNIRS was the only method that classified game demand at higher than chance levels (accuracy = 0.66, F1 = 0.68). It was not possible to classify pain vs. no-pain at higher than chance level. The results demonstrate the viability of utilising fNIRS data to classify levels of game demand and the difficulty of classifying pain when another task is present.

Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e., optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky-Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.

See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions

Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.

Insights from a laboratory and naturalistic investigation on stress, rumination and frontal brain functioning in MDD: An fNIRS study

Recent research has emphasized rumination as an important maintaining factor in various mental disorders. However, operationalization and therefore induction of rumination in experimental settings poses a major challenge in terms of ecological validity. As stress seems to play a key role in everyday situations eliciting rumination, we conducted two stress paradigms while assessing behavioral and neurophysiological measures. Aiming to replicate previous findings on induced rumination by means of the Trier Social Stress Test (TSST) and comparing them to physiological (pain) stress, a clinical sample of patients with Major Depressive Disorder (MDD; n = 22) and healthy controls (HC; n = 23) was recruited. Cortical blood oxygenation was assessed during the stress paradigms using functional near-infrared spectroscopy (fNIRS). Further, we used ecological momentary assessment (EMA) of stress, rumination and mood to be able to correlate ruminative responses during induced stress and everyday rumination. Our results showed that social stress but not physiological stress induced depressive rumination in MDD but not in HC. Further, rumination reactivity in response to social stress but not to physiological stress was significantly associated with rumination reactivity in everyday life as assessed with EMA. With respect to cortical oxygenation, MDD subjects showed hypoactivity in the Cognitive Control Network during the TSST, which mediated the differences between MDD and HC in post-stress rumination. Our findings emphasize the role of negative social triggers in depressive rumination and validate the TSST as an induction method for depressive rumination. The results inform future developments in psychotherapeutic treatment for depressive rumination.

Non-Invasive Functional Evaluation of the Human Spinal Cord by Assessing the Peri-Spinal Neurovascular Network With Near Infrared Spectroscopy

Current medical care lacks an effective functional evaluation for the spinal cord. Magnetic resonance imaging and computed tomography mainly provide structural information of the spinal cord, while spinal somatosensory evoked potentials are limited by a low signal to noise ratio. We developed a non-invasive approach based on near-infrared spectroscopy in dual-wavelength (760 and 850 nm for deoxy- or oxyhemoglobin respectively) to record the neurovascular response (NVR) of the peri-spinal vascular network at the 7th cervical and 10th thoracic vertebral levels of the spinal cord, triggered by unilateral median nerve electrical stimulation (square pulse, 5-10 mA, 5 ms, 1 pulse every 4 minutes) at the wrist. Amplitude, rise-time, and duration of NVR were characterized in 20 healthy participants. A single, painless stimulus was able to elicit a high signal-to-noise ratio and multi-segmental NVR (mainly from Oxyhemoglobin) with a fast rise time of 6.18 [4.4-10.4] seconds (median [Percentile 25-75]) followed by a slow decay phase for about 30 seconds toward the baseline. Cervical NVR was earlier and larger than thoracic and no left/right asymmetry was detected. Stimulus intensity/NVR amplitude fitted to a 2nd order function. The characterization and feasibility of the peri-spinal NVR strongly support the potential clinical applications for a functional assessment of spinal cord lesions.

Sleep deprivation impairs cognitive performance, alters task-associated cerebral blood flow and decreases cortical neurovascular coupling-related hemodynamic responses

Sleep deprivation (SD) is a common condition and an important health concern. In addition to metabolic and cardiovascular risks, SD associates with decreases in cognitive performance. Neurovascular coupling (NVC, "functional hyperemia") is a critical homeostatic mechanism, which maintains adequate blood supply to the brain during periods of intensive neuronal activity. To determine whether SD alters NVC responses and cognitive performance, cognitive and hemodynamic NVC assessments were conducted prior to and 24 h post-SD in healthy young male individuals (n = 10, 27 ± 3 years old). Cognition was evaluated with a battery of tests from the Cambridge Neuropsychological Test Automated Battery (CANTAB). Hemodynamic components of NVC were measured by transcranial Doppler sonography (TCD) during cognitive stimulation, dynamic retinal vessel analysis (DVA) during flicker light stimulation, and functional near infrared spectroscopy (fNIRS) during finger tapping motor task. Cognitive assessments revealed impairments in reaction time and sustained attention after 24 h of SD. Functional NIRS analysis revealed that SD significantly altered hemodynamic responses in the prefrontal cortex and somatosensory cortex during a motor task. NVC-related vascular responses measured by DVA and TCD did not change significantly. Interestingly, TCD detected decreased task-associated cerebral blood flow (CBF) in the right middle cerebral artery in sleep deprived participants. Our results demonstrate that 24 h of SD lead to impairments in cognitive performance together with altered CBF and hemodynamic components of cortical NVC responses.

Brain–Computer Interfacing Using Functional Near-Infrared Spectroscopy (fNIRS)

Functional Near-Infrared Spectroscopy (fNIRS) is a wearable optical spectroscopy system originally developed for continuous and non-invasive monitoring of brain function by measuring blood oxygen concentration. Recent advancements in brain–computer interfacing allow us to control the neuron function of the brain by combining it with fNIRS to regulate cognitive function. In this review manuscript, we provide information regarding current advancement in fNIRS and how it provides advantages in developing brain–computer interfacing to enable neuron function. We also briefly discuss about how we can use this technology for further applications.

Prefrontal Cortex Activation During Verbal Fluency Task and Tower of London Task in Schizophrenia and Major Depressive Disorder

Background: Cognitive dysfunction is a common clinical feature of mental disorders. A number of functional near-infrared spectroscopy (fNIRS) studies have shown reduced prefrontal activation during the verbal fluency task (VFT) in schizophrenia (SZ) and major depressive disorder (MDD). However, no studies have examined and compared the brain activation patterns during the Tower of London (TOL), which is another classic, high-sensitivity executive function testing tool, in these two serious mental disorders. This study aimed to assess the characteristics of brain activation during the two different cognitive tasks in SZ and MDD patients. Methods: This study recruited 30 patients with SZ, 30 patients with MDD, and 30 demographically matched healthy controls (HCs). The hemodynamic changes of the prefrontal cortex (PFC) were measured using 32-channel fNIRS during performance of the TOL task and VFT task. Results: SZ patients showed poorer VFT performance than MDD patients and HCs, and the two patient groups showed poorer TOL performance than HCs. Compared to HCs, both of the patient groups exhibited a significant decreased activation in the extensive PFC. Particularly in certain channels in the dorsolateral PFC (DLPFC), SZ patients exhibited significantly decreased hemodynamic changes than the MDD patients. Conclusions: Patients with SZ and MDD have different levels of impairment in different cognitive domains and different patterns of brain activation during the two cognitive tasks. Further research is needed to determine the use of fNIRS for clinical evaluation and diagnosis.

Most favorable stimulation duration in the sensorimotor cortex for fNIRS-based BCI

One of the primary objectives of the brain-computer interface (BCI) is to obtain a command with higher classification accuracy within the shortest possible time duration. Therefore, this study evaluates several stimulation durations to propose a duration that can yield the highest classification accuracy. Furthermore, this study aims to address the inherent delay in the hemodynamic responses (HRs) for the command generation time. To this end, HRs in the sensorimotor cortex were evaluated for the functional near-infrared spectroscopy (fNIRS)-based BCI. To evoke brain activity, right-hand-index finger poking and tapping tasks were used. In this study, six different stimulation durations (i.e., 1, 3, 5, 7, 10, and 15 s) were tested on 10 healthy male subjects. Upon stimulation, different temporal features and multiple time windows were utilized to extract temporal features. The extracted features were then classified using linear discriminant analysis. The classification results using the main HR showed that a 5 s stimulation duration could yield the highest classification accuracy, i.e., 74%, with a combination of the mean and maximum value features. However, the results were not significantly different from the classification accuracy obtained using the 15 s stimulation. To further validate the results, a classification using the initial dip was performed. The results obtained endorsed the finding with an average classification accuracy of 73.5% using the features of minimum peak and skewness in the 5 s window. The results based on classification using the initial dip for 5 s were significantly different from all other tested stimulation durations (p < 0.05) for all feature combinations. Moreover, from the visual inspection of the HRs, it is observed that the initial dip occurred as soon as the task started, but the main HR had a delay of more than 2 s. Another interesting finding is that impulsive stimulation in the sensorimotor cortex can result in the generation of a clearer initial dip phenomenon. The results reveal that the command for the fNIRS-based BCI can be generated using the 5 s stimulation duration. In conclusion, the use of the initial dip can reduce the time taken for the generation of commands and can be used to achieve a higher classification accuracy for the fNIRS-BCI within a 5 s task duration rather than relying on longer durations.

Comparison of functional activation responses from the auditory cortex derived using multi-distance frequency domain and continuous wave near-infrared spectroscopy

Significance: Quantitative measurements of cerebral hemodynamic changes due to functional activation are widely accomplished with commercial continuous wave (CW-NIRS) instruments despite the availability of the more rigorous multi-distance frequency domain (FD-NIRS) approach. A direct comparison of the two approaches to functional near-infrared spectroscopy can help in the interpretation of optical data and guide implementations of diffuse optical instruments for measuring functional activation. Aim: We explore the differences between CW-NIRS and multi-distance FD-NIRS by comparing measurements of functional activation in the human auditory cortex. Approach: Functional activation of the human auditory cortex was measured using a commercial frequency domain near-infrared spectroscopy instrument for 70 dB sound pressure level broadband noise and pure tone (1000 Hz) stimuli. Changes in tissue oxygenation were calculated using the modified Beer-Lambert law (CW-NIRS approach) and the photon diffusion equation (FD-NIRS approach). Results: Changes in oxygenated hemoglobin measured with the multi-distance FD-NIRS approach were about twice as large as those measured with the CW-NIRS approach. A finite-element simulation of the functional activation problem was performed to demonstrate that tissue oxygenation changes measured with the CW-NIRS approach is more accurate than that with multi-distance FD-NIRS. Conclusions: Multi-distance FD-NIRS approaches tend to overestimate functional activation effects, in part due to partial volume effects.

A Novel Spectrum Contrast Mapping Method for Functional Magnetic Resonance Imaging Data Analysis

Many studies reported that spontaneous fluctuation of the blood oxygen level-dependent signal exists in multiple frequency components and changes over time. By assuming a reliable energy contrast between low- and high-frequency bands for each voxel, we developed a novel spectrum contrast mapping (SCM) method to decode brain activity at the voxel-wise level and further validated it in designed experiments. SCM consists of the following steps: first, the time course of each given voxel is subjected to fast Fourier transformation; the corresponding spectrum is divided into low- and high-frequency bands by given reference frequency points; then, the spectral energy ratio of the low- to high-frequency bands is calculated for each given voxel. Finally, the activity decoding map is formed by the aforementioned energy contrast values of each voxel. Our experimental results demonstrate that the SCM (1) was able to characterize the energy contrast of task-related brain regions; (2) could decode brain activity at rest, as validated by the eyes-closed and eyes-open resting-state experiments; (3) was verified with test-retest validation, indicating excellent reliability with most coefficients > 0.9 across the test sessions; and (4) could locate the aberrant energy contrast regions which might reveal the brain pathology of brain diseases, such as Parkinson’s disease. In summary, we demonstrated that the reliable energy contrast feature was a useful biomarker in characterizing brain states, and the corresponding SCM showed excellent brain activity-decoding performance at the individual and group levels, implying its potentially broad application in neuroscience, neuroimaging, and brain diseases.

Current Review of Optical Neural Interfaces for Clinical Applications

Neural interfaces, which enable the recording and stimulation of living neurons, have emerged as valuable tools in understanding the brain in health and disease, as well as serving as neural prostheses. While neural interfaces are typically based on electrical transduction, alternative energy modalities have been explored to create safe and effective approaches. Among these approaches, optical methods of linking neurons to the outside world have gained attention because light offers high spatial selectivity and decreased invasiveness. Here, we review the current state-of-art of optical neural interfaces and their clinical applications. Optical neural interfaces can be categorized into optical control and optical readout, each of which can be divided into intrinsic and extrinsic approaches. We discuss the advantages and disadvantages of each of these methods and offer a comparison of relative performance. Future directions, including their clinical opportunities, are discussed with regard to the optical properties of biological tissue.

Conditional-GAN Based Data Augmentation for Deep Learning Task Classifier Improvement Using fNIRS Data

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique used for mapping the functioning human cortex. fNIRS can be widely used in population studies due to the technology’s economic, non-invasive, and portable nature. fNIRS can be used for task classification, a crucial part of functioning with Brain-Computer Interfaces (BCIs). fNIRS data are multidimensional and complex, making them ideal for deep learning algorithms for classification. Deep Learning classifiers typically need a large amount of data to be appropriately trained without over-fitting. Generative networks can be used in such cases where a substantial amount of data is required. Still, the collection is complex due to various constraints. Conditional Generative Adversarial Networks (CGAN) can generate artificial samples of a specific category to improve the accuracy of the deep learning classifier when the sample size is insufficient. The proposed system uses a CGAN with a CNN classifier to enhance the accuracy through data augmentation. The system can determine whether the subject’s task is a Left Finger Tap, Right Finger Tap, or Foot Tap based on the fNIRS data patterns. The authors obtained a task classification accuracy of 96.67% for the CGAN-CNN combination.

The Potential Role of fNIRS in Evaluating Levels of Consciousness

Over the last few decades, neuroimaging techniques have transformed our understanding of the brain and the effect of neurological conditions on brain function. More recently, light-based modalities such as functional near-infrared spectroscopy have gained popularity as tools to study brain function at the bedside. A recent application is to assess residual awareness in patients with disorders of consciousness, as some patients retain awareness albeit lacking all behavioural response to commands. Functional near-infrared spectroscopy can play a vital role in identifying these patients by assessing command-driven brain activity. The goal of this review is to summarise the studies reported on this topic, to discuss the technical and ethical challenges of working with patients with disorders of consciousness, and to outline promising future directions in this field.

Optimization of Task Allocation for Collaborative Brain-Computer Interface Based on Motor Imagery

OBJECTIVE

Collaborative brain-computer interfaces (cBCIs) can make the BCI output more credible by jointly decoding concurrent brain signals from multiple collaborators. Current cBCI systems usually require all collaborators to execute the same mental tasks (common-work strategy). However, it is still unclear whether the system performance will be improved by assigning different tasks to collaborators (division-of-work strategy) while keeping the total tasks unchanged. Therefore, we studied a task allocation scheme of division-of-work and compared the corresponding classification accuracies with common-work strategy's.

APPROACH

This study developed an electroencephalograph (EEG)-based cBCI which had six instructions related to six different motor imagery tasks (MI-cBCI), respectively. For the common-work strategy, all five subjects as a group had the same whole instruction set and they were required to conduct the same instruction at a time. For the division-of-work strategy, every subject's instruction set was a subset of the whole one and different from each other. However, their union set was equal to the whole set. Based on the number of instructions in a subset, we divided the division-of-work strategy into four types, called "2 Tasks" … "5 Tasks." To verify the effectiveness of these strategies, we employed EEG data collected from 19 subjects who independently performed six types of MI tasks to conduct the pseudo-online classification of MI-cBCI.

MAIN RESULTS

Taking the number of tasks performed by one collaborator as the horizontal axis (two to six), the classification accuracy curve of MI-cBCI was mountain-like. The curve reached its peak at "4 Tasks," which means each subset contained four instructions. It outperformed the common-work strategy ("6 Tasks") in classification accuracy (72.29 ± 4.43 vs. 58.53 ± 4.36%).

SIGNIFICANCE

The results demonstrate that our proposed task allocation strategy effectively enhanced the cBCI classification performance and reduced the individual workload.

Functional Brain Imaging Reliably Predicts Bimanual Motor Skill Performance in a Standardized Surgical Task

Currently, there is a dearth of objective metrics for assessing bi-manual motor skills, which are critical for high-stakes professions such as surgery. Recently, functional near-infrared spectroscopy (fNIRS) has been shown to be effective at classifying motor task types, which can be potentially used for assessing motor performance level. In this work, we use fNIRS data for predicting the performance scores in a standardized bi-manual motor task used in surgical certification and propose a deep-learning framework 'Brain-NET' to extract features from the fNIRS data. Our results demonstrate that the Brain-NET is able to predict bi-manual surgical motor skills based on neuroimaging data accurately ( R2=0.73). Furthermore, the classification ability of the Brain-NET model is demonstrated based on receiver operating characteristic (ROC) curves and area under the curve (AUC) values of 0.91. Hence, these results establish that fNIRS associated with deep learning analysis is a promising method for a bedside, quick and cost-effective assessment of bi-manual skill levels.

Recognition of Flexion and Extension Imagery Involving the Right and Left Arms Based on Deep Belief Network and Functional Near-Infrared Spectroscopy

Brain-computer interaction based on motor imagery (MI) is an important brain-computer interface (BCI). Most methods for MI classification are based on electroencephalogram (EEG), and few studies have investigated signal processing based on MI-Functional Near-Infrared Spectroscopy (fNIRS). In addition, there is a need to improve the classification accuracy for MI fNIRS methods. In this study, a deep belief network (DBN) based on a restricted Boltzmann machine (RBM) was used to classify fNIRS signals of flexion and extension imagery involving the left and right arms. fNIRS signals from 16 channels covering the motor cortex area were recorded for each of 10 subjects executing or imagining flexion and extension involving the left and right arms. Oxygenated hemoglobin (HbO) concentration was used as a feature to train two RBMs that were subsequently stacked with an additional softmax regression output layer to construct DBN. We also explored the DBN model classification accuracy for the test dataset from one subject using training dataset from other subjects. The average DBN classification accuracy for flexion and extension movement and imagery involving the left and right arms was 84.35 ± 3.86% and 78.19 ± 3.73%, respectively. For a given DBN model, better classification results are obtained for test datasets for a given subject when the model is trained using dataset from the same subject than when the model is trained using datasets from other subjects. The results show that the DBN algorithm can effectively identify flexion and extension imagery involving the right and left arms using fNIRS. This study is expected to serve as a reference for constructing online MI-BCI systems based on DBN and fNIRS.

Decoding different working memory states during an operation span task from prefrontal fNIRS signals

We propose an effective and practical decoding method of different mental states for potential applications for the design of brain-computer interfaces, prediction of cognitive behaviour, and investigation of cognitive mechanism. Functional near infrared spectroscopy (fNIRS) signals that interrogated the prefrontal and parietal cortices and were evaluated by generalized linear model were recorded when nineteen healthy adults performed the operation span (OSPAN) task. The oxygenated hemoglobin changes during OSPAN, response, and rest periods were classified with a support vector machine (SVM). The relevance vector regression algorithm was utilized for prediction of cognitive performance based on multidomain features of fNIRS signals from the OSPAN task. We acquired decent classification accuracies for OSPAN vs. response (above 91.2%) and for OSPAN vs. rest (above 94.7%). Eight of the ten cognitive testing scores could be predicted from the combination of OSPAN and response features, which indicated the brain hemodynamic responses contain meaningful information suitable for predicting cognitive performance.

Prefrontal Cortex Activation During Motor Sequence Learning Under Interleaved and Repetitive Practice: A Two-Channel Near-Infrared Spectroscopy Study

Training under high interference conditions through interleaved practice (IP) results in performance suppression during training but enhances long-term performance relative to repetitive practice (RP) involving low interference. Previous neuroimaging work addressing this contextual interference effect of motor learning has relied heavily on the blood-oxygen-level-dependent (BOLD) response using functional magnetic resonance imaging (fMRI) methodology resulting in mixed reports of prefrontal cortex (PFC) recruitment under IP and RP conditions. We sought to clarify these equivocal findings by imaging bilateral PFC recruitment using functional near-infrared spectroscopy (fNIRS) while discrete key pressing sequences were trained under IP and RP schedules and subsequently tested following a 24-h delay. An advantage of fNIRS over the fMRI BOLD response is that the former measures oxygenated and deoxygenated hemoglobin changes independently allowing for assessment of cortical hemodynamics even when there is neurovascular decoupling. Despite slower sequence performance durations under IP, bilateral PFC oxygenated and deoxygenated hemoglobin values did not differ between practice conditions. During test, however, slower performance from those previously trained under RP coincided with hemispheric asymmetry in PFC recruitment. Specifically, following RP, test deoxygenated hemoglobin values were significantly lower in the right PFC. The present findings contrast with previous behavioral demonstrations of increased cognitive demand under IP to illustrate a more complex involvement of the PFC in the contextual interference effect. IP and RP incur similar levels of bilateral PFC recruitment, but the processes underlying the recruitment are dissimilar. PFC recruitment during IP supports action reconstruction and memory elaboration while RP relies on PFC recruitment to maintain task variation information in working memory from trial to trial. While PFC recruitment under RP serves to enhance immediate performance, it does not support long-term performance.

Data Processing in Functional Near-Infrared Spectroscopy (fNIRS) Motor Control Research

FNIRS pre-processing and processing methodologies are very important-how a researcher chooses to process their data can change the outcome of an experiment. The purpose of this review is to provide a guide on fNIRS pre-processing and processing techniques pertinent to the field of human motor control research. One hundred and twenty-three articles were selected from the motor control field and were examined on the basis of their fNIRS pre-processing and processing methodologies. Information was gathered about the most frequently used techniques in the field, which included frequency cutoff filters, wavelet filters, smoothing filters, and the general linear model (GLM). We discuss the methodologies of and considerations for these frequently used techniques, as well as those for some alternative techniques. Additionally, general considerations for processing are discussed.

Functional interferometric diffusing wave spectroscopy of the human brain

Cerebral blood flow (CBF) is essential for brain function, and CBF-related signals can inform us about brain activity. Yet currently, high-end medical instrumentation is needed to perform a CBF measurement in adult humans. Here, we describe functional interferometric diffusing wave spectroscopy (fiDWS), which introduces and collects near-infrared light via the scalp, using inexpensive detector arrays to rapidly monitor coherent light fluctuations that encode brain blood flow index (BFI), a surrogate for CBF. Compared to other functional optical approaches, fiDWS measures BFI faster and deeper while also providing continuous wave absorption signals. Achieving clear pulsatile BFI waveforms at source-collector separations of 3.5 cm, we confirm that optical BFI, not absorption, shows a graded hypercapnic response consistent with human cerebrovascular physiology, and that BFI has a better contrast-to-noise ratio than absorption during brain activation. By providing high-throughput measurements of optical BFI at low cost, fiDWS will expand access to CBF.

Exploring the SenseMaking Process through Interactions and fNIRS in Immersive Visualization

Theories of cognition inform our decisions when designing human-computer interfaces, and immersive systems enable us to examine these theories. This work explores the sensemaking process in an immersive environment through studying both internal and external user behaviors with a classical visualization problem: a visual comparison and clustering task. We developed an immersive system to perform a user study, collecting user behavior data from different channels: AR HMD for capturing external user interactions, functional near-infrared spectroscopy (fNIRS) for capturing internal neural sequences, and video for references. To examine sensemaking, we assessed how the layout of the interface (planar 2D vs. cylindrical 3D layout) and the challenge level of the task (low vs. high cognitive load) influenced the users' interactions, how these interactions changed over time, and how they influenced task performance. We also developed a visualization system to explore joint patterns among all the data channels. We found that increased interactions and cerebral hemodynamic responses were associated with more accurate performance, especially on cognitively demanding trials. The layout types did not reliably influence interactions or task performance. We discuss how these findings inform the design and evaluation of immersive systems, predict user performance and interaction, and offer theoretical insights about sensemaking from the perspective of embodied and distributed cognition.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Effect of force accuracy on hemodynamic response: an fNIRS study using fine visuomotor task

Objective. Despite converging neuroimaging studies investigating how neural activity is modulated by various motor related factors, such as movement velocity and force magnitude, little has been devoted to identifying the effect of force accuracy. This study thus aimed to investigate the effect of task difficulty on cortical neural responses when participants performed a visuomotor task with varying demands on force accuracy.Approach. Fourteen healthy adults performed a set of force generation operations with six levels of force accuracy. The participants held a pen-shaped tool and moved the tool along a planar ring path, meanwhile producing a constant force against the plane under visual guidance. The required force accuracy was modulated by allowable tolerance of the force during the task execution. We employed functional near-infrared spectroscopy to record signals from bilateral prefrontal, sensorimotor and occipital areas, used the hemoglobin concentration as indicators of cortical activation, then calculated the effective connectivity across these regions by Granger causality.Main results.We observed overall stronger activation (oxy-hemoglobin concentration,p= 0.015) and connectivity (p< 0.05) associated with the initial increase in force accuracy, and the diminished trend in activation and connectivity when participants were exposed to excessive demands on accurate force generation. These findings suggested that the increasing task difficulty would be only beneficial for the mental investment up to a certain point, and above that point neural responses would show patterns of lower activation and connections, revealing mental overload at excessive task demands.Significance.Our results provide the first evidence for the inverted U-shaped effect of force accuracy on hemodynamic responses during fine visuomotor tasks. The insights obtained through this study also highlight the essential role of inter-region connectivity alterations for coping with task difficulty, enhance our understanding of the underlying motor neural processes, and provide the groundwork for developing adaptive neurorehabilitation strategies.

CNN-based classification of fNIRS signals in motor imagery BCI system

Objective. Development of a brain-computer interface (BCI) requires classification of brain neural activities to different states. Functional near-infrared spectroscopy (fNIRS) can measure the brain activities and has great potential for BCI. In recent years, a large number of classification algorithms have been proposed, in which deep learning methods, especially convolutional neural network (CNN) methods are successful. fNIRS signal has typical time series properties, we combined fNIRS data and kinds of CNN-based time series classification (TSC) methods to classify BCI task.Approach. In this study, participants were recruited for a left and right hand motor imagery experiment and the cerebral neural activities were recorded by fNIRS equipment (FOIRE-3000). TSC methods are used to distinguish the brain activities when imagining the left or right hand. We have tested the overall person, single person and overall person with single-channel classification results, and these methods achieved excellent classification results. We also compared the CNN-based TSC methods with traditional classification methods such as support vector machine.Main results. Experiments showed that the CNN-based methods have significant advantages in classification accuracy: the CNN-based methods have achieved remarkable results in the classification of left-handed and right-handed imagination tasks, reaching 98.6% accuracy on overall person, 100% accuracy on single person, and in the single-channel classification an accuracy of 80.1% has been achieved with the best-performing channel.Significance. These results suggest that using the CNN-based TSC methods can significantly improve the BCI performance and also lay the foundation for the miniaturization and portability of training rehabilitation equipment.

Guiding functional near-infrared spectroscopy optode-layout design using individual (f)MRI data: effects on signal strength

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain-computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.

Evidence of Neurovascular Un-Coupling in Mild Alzheimer's Disease through Multimodal EEG-fNIRS and Multivariate Analysis of Resting-State Data

Alzheimer’s disease (AD) is associated with modifications in cerebral blood perfusion and autoregulation. Hence, neurovascular coupling (NC) alteration could become a biomarker of the disease. NC might be assessed in clinical settings through multimodal electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Multimodal EEG-fNIRS was recorded at rest in an ambulatory setting to assess NC and to evaluate the sensitivity and specificity of the methodology to AD. Global NC was evaluated with a general linear model (GLM) framework by regressing whole-head EEG power envelopes in three frequency bands (theta, alpha and beta) with average fNIRS oxy- and deoxy-hemoglobin concentration changes in the frontal and prefrontal cortices. NC was lower in AD compared to healthy controls (HC) with significant differences in the linkage of theta and alpha bands with oxy- and deoxy-hemoglobin, respectively (p = 0.028 and p = 0.020). Importantly, standalone EEG and fNIRS metrics did not highlight differences between AD and HC. Furthermore, a multivariate data-driven analysis of NC between the three frequency bands and the two hemoglobin species delivered a cross-validated classification performance of AD and HC with an Area Under the Curve, AUC = 0.905 (p = 2.17 × 10⁻⁵). The findings demonstrate that EEG-fNIRS may indeed represent a powerful ecological tool for clinical evaluation of NC and early identification of AD.

Systemic Review on Transcranial Electrical Stimulation Parameters and EEG/fNIRS Features for Brain Diseases

Background

Brain disorders are gradually becoming the leading cause of death worldwide. However, the lack of knowledge of brain disease’s underlying mechanisms and ineffective neuropharmacological therapy have led to further exploration of optimal treatments and brain monitoring techniques.

Objective

This study aims to review the current state of brain disorders, which utilize transcranial electrical stimulation (tES) and daily usable noninvasive neuroimaging techniques. Furthermore, the second goal of this study is to highlight available gaps and provide a comprehensive guideline for further investigation.

Method

A systematic search was conducted of the PubMed and Web of Science databases from January 2000 to October 2020 using relevant keywords. Electroencephalography (EEG) and functional near-infrared spectroscopy were selected as noninvasive neuroimaging modalities. Nine brain disorders were investigated in this study, including Alzheimer’s disease, depression, autism spectrum disorder, attention-deficit hyperactivity disorder, epilepsy, Parkinson’s disease, stroke, schizophrenia, and traumatic brain injury.

Results

Sixty-seven studies (1,385 participants) were included for quantitative analysis. Most of the articles (82.6%) employed transcranial direct current stimulation as an intervention method with modulation parameters of 1 mA intensity (47.2%) for 16–20 min (69.0%) duration of stimulation in a single session (36.8%). The frontal cortex (46.4%) and the cerebral cortex (47.8%) were used as a neuroimaging modality, with the power spectrum (45.7%) commonly extracted as a quantitative EEG feature.

Conclusion

An appropriate stimulation protocol applying tES as a therapy could be an effective treatment for cognitive and neurological brain disorders. However, the optimal tES criteria have not been defined; they vary across persons and disease types. Therefore, future work needs to investigate a closed-loop tES with monitoring by neuroimaging techniques to achieve personalized therapy for brain disorders.

Defining Surgical Terminology and Risk for Brain Computer Interface Technologies

With the emergence of numerous brain computer interfaces (BCI), their form factors, and clinical applications the terminology to describe their clinical deployment and the associated risk has been vague. The terms “minimally invasive” or “non-invasive” have been commonly used, but the risk can vary widely based on the form factor and anatomic location. Thus, taken together, there needs to be a terminology that best accommodates the surgical footprint of a BCI and their attendant risks. This work presents a semantic framework that describes the BCI from a procedural standpoint and its attendant clinical risk profile. We propose extending the common invasive/non-invasive distinction for BCI systems to accommodate three categories in which the BCI anatomically interfaces with the patient and whether or not a surgical procedure is required for deployment: (1) Non-invasive—BCI components do not penetrate the body, (2) Embedded—components are penetrative, but not deeper than the inner table of the skull, and (3) Intracranial –components are located within the inner table of the skull and possibly within the brain volume. Each class has a separate risk profile that should be considered when being applied to a given clinical population. Optimally, balancing this risk profile with clinical need provides the most ethical deployment of these emerging classes of devices. As BCIs gain larger adoption, and terminology becomes standardized, having an improved, more precise language will better serve clinicians, patients, and consumers in discussing these technologies, particularly within the context of surgical procedures.

Investigating the Viability of Motor Imagery as a Physical Rehabilitation Treatment for Patients With Stroke-Induced Motor Cortical Damage

Although around 83% of individuals survive a stroke, they usually experience a significant loss in their motor execution (ME) capabilities due to their acquired cortical infarction. The loss of significant ME capabilities due to stroke damage was previously thought to be irreversible. Active movement therapies show considerable promise but depend on motor performance, excluding many otherwise eligible patients. Motor imagery (MI), a process that involves the use of mirror neurons to imagine motor activity, has emerged as a possible avenue to re-acquire some physical abilities lost to stroke damage. This paper examines previous studies to compare the strength of brain activation and connectivity in individuals who have brain lesions and those who do not as they all attempt ME and MI tasks. This paper reviews case studies investigating the direct effect of motor imagery in conjunction with physical therapy and the limitations of motor imagery based on the location of cortical damage and other variables, such as age. The findings analyzed in this review indicate that MI would serve as a beneficial addition to physical therapy and a viable option to stimulate motor evoked potentials (MEPs) in individuals not capable of pursuing physical therapy due to severe motor impairment. Regardless of the presence of brain lesions, motor imagery has consistently had a positive impact on motor rehabilitation either in boosting treatment or stimulating neuromuscular pathways. Therefore, we have concluded that MI is a viable supplemental treatment plan for motor recovery in most patients with motor cortical atrophy.

Motor Training Using Mental Workload (MWL) With an Assistive Soft Exoskeleton System: A Functional Near-Infrared Spectroscopy (fNIRS) Study for Brain-Machine Interface (BMI)

Mental workload is a neuroergonomic human factor, which is widely used in planning a system's safety and areas like brain-machine interface (BMI), neurofeedback, and assistive technologies. Robotic prosthetics methodologies are employed for assisting hemiplegic patients in performing routine activities. Assistive technologies' design and operation are required to have an easy interface with the brain with fewer protocols, in an attempt to optimize mobility and autonomy. The possible answer to these design questions may lie in neuroergonomics coupled with BMI systems. In this study, two human factors are addressed: designing a lightweight wearable robotic exoskeleton hand that is used to assist the potential stroke patients with an integrated portable brain interface using mental workload (MWL) signals acquired with portable functional near-infrared spectroscopy (fNIRS) system. The system may generate command signals for operating a wearable robotic exoskeleton hand using two-state MWL signals. The fNIRS system is used to record optical signals in the form of change in concentration of oxy and deoxygenated hemoglobin (HbO and HbR) from the pre-frontal cortex (PFC) region of the brain. Fifteen participants participated in this study and were given hand-grasping tasks. Two-state MWL signals acquired from the PFC region of the participant's brain are segregated using machine learning classifier-support vector machines (SVM) to utilize in operating a robotic exoskeleton hand. The maximum classification accuracy is 91.31%, using a combination of mean-slope features with an average information transfer rate (ITR) of 1.43. These results show the feasibility of a two-state MWL (fNIRS-based) robotic exoskeleton hand (BMI system) for hemiplegic patients assisting in the physical grasping tasks.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Based on Convolutional Neural Networks

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain–computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

High density optical neuroimaging predicts surgeons's subjective experience and skill levels

Measuring cognitive load is important for surgical education and patient safety. Traditional approaches of measuring cognitive load of surgeons utilise behavioural metrics to measure performance and surveys and questionnaires to collect reports of subjective experience. These have disadvantages such as sporadic data, occasionally intrusive methodologies, subjective or misleading self-reporting. In addition, traditional approaches use subjective metrics that cannot distinguish between skill levels. Functional neuroimaging data was collected using a high density, wireless NIRS device from sixteen surgeons (11 attending surgeons and 5 surgery resident) and 17 students while they performed two laparoscopic tasks (Peg transfer and String pass). Participant’s subjective mental load was assessed using the NASA-TLX survey. Machine learning approaches were used for predicting the subjective experience and skill levels. The Prefrontal cortex (PFC) activations were greater in students who reported higher-than-median task load, as measured by the NASA-TLX survey. However in the case of attending surgeons the opposite tendency was observed, namely higher activations in the lower v higher task loaded subjects. We found that response was greater in the left PFC of students particularly near the dorso- and ventrolateral areas. We quantified the ability of PFC activation to predict the differences in skill and task load using machine learning while focussing on the effects of NIRS channel separation distance on the results. Our results showed that the classification of skill level and subjective task load could be predicted based on PFC activation with an accuracy of nearly 90%. Our finding shows that there is sufficient information available in the optical signals to make accurate predictions about the surgeons’ subjective experiences and skill levels. The high accuracy of results is encouraging and suggest the integration of the strategy developed in this study as a promising approach to design automated, more accurate and objective evaluation methods.