fNIRS-based brain-computer interfaces: a review

Assessing Time-Resolved fNIRS for Brain-Computer Interface Applications of Mental Communication

Brain-computer interfaces (BCIs) are becoming increasingly popular as a tool to improve the quality of life of patients with disabilities. Recently, time-resolved functional near-infrared spectroscopy (TR-fNIRS) based BCIs are gaining traction because of their enhanced depth sensitivity leading to lower signal contamination from the extracerebral layers. This study presents the first account of TR-fNIRS based BCI for “mental communication” on healthy participants. Twenty-one (21) participants were recruited and were repeatedly asked a series of questions where they were instructed to imagine playing tennis for “yes” and to stay relaxed for “no.” The change in the mean time-of-flight of photons was used to calculate the change in concentrations of oxy- and deoxyhemoglobin since it provides a good compromise between depth sensitivity and signal-to-noise ratio. Features were extracted from the average oxyhemoglobin signals to classify them as “yes” or “no” responses. Linear-discriminant analysis (LDA) and support vector machine (SVM) classifiers were used to classify the responses using the leave-one-out cross-validation method. The overall accuracies achieved for all participants were 75% and 76%, using LDA and SVM, respectively. The results also reveal that there is no significant difference in accuracy between questions. In addition, physiological parameters [heart rate (HR) and mean arterial pressure (MAP)] were recorded on seven of the 21 participants during motor imagery (MI) and rest to investigate changes in these parameters between conditions. No significant difference in these parameters was found between conditions. These findings suggest that TR-fNIRS could be suitable as a BCI for patients with brain injuries.

Effects of Processing Methods on fNIRS Signals Assessed During Active Walking Tasks in Older Adults

Functional near infrared spectroscopy (fNIRS) is a noninvasive optics-based neuroimaging modality successfully applied to real-life settings. The technology uses light in the near infrared range (650-950nm) to track changes in oxygenated (HbO2) and deoxygenated hemoglobin (Hb) obtained from measured light intensity using light-tissue interaction principles. fNIRS data processing involves artifact removal and hemodynamic signal conversion using modified Beer-Lambert law (MBLL) to obtain Hb and HbO2, reliably. fNIRS signals can get contaminated by various noise sources of physiological and non-physiological origins. Various algorithms have been proposed for the elimination of artifacts from frequency selective filters to blind source separation methods. Hemodynamic signal extraction using raw intensity measurements at multiple wavelengths based on MBLL usually involves apriori knowledge of certain conversion parameters such as molar extinction coefficients ( ε ) and differential path length factor (DPF). Different sets of conversion parameters dependent upon wavelength, chromophores, and age have been reported. Variation in processing algorithms and parameters can cause differences in Hb and HbO2 extraction which can in turn change study outcomes. Using fNIRS, we have previously shown significant increases in oxygenation in the prefrontal cortex from Single-Task-Walking (STW) to Dual-task-Walking (DTW) conditions in older adults due to greater cognitive demands inherent in the latter condition. In the current study, we re-analyzed our data and determined that although using different conversion parameters i.e. ε and age dependent DPF and filter cut-off frequencies at 0.14 and 0.08Hz including those designed to remove confounding effects of Mayer waves had caused some linear increases or decreases on the extracted Hb and HbO2 signals, those effects were minimal in task related comparisons and hence, the overall study outcomes.

Indoor Simulated Training Environment for Brain-Controlled Wheelchair Based on Steady-State Visual Evoked Potentials

Brain-controlled wheelchair (BCW) has the potential to improve the quality of life for people with motor disabilities. A lot of training is necessary for users to learn and improve BCW control ability and the performances of BCW control are crucial for patients in daily use. In consideration of safety and efficiency, an indoor simulated training environment is built up in this paper to improve the performance of BCW control. The indoor simulated environment mainly realizes BCW implementation, simulated training scenario setup, path planning and recommendation, simulated operation, and scoring. And the BCW is based on steady-state visual evoked potentials (SSVEP) and the filter bank canonical correlation analysis (FBCCA) is used to analyze the electroencephalography (EEG). Five tasks include individual accuracy, simple linear path, obstacles avoidance, comprehensive steering scenarios, and evaluation task are designed, 10 healthy subjects were recruited and carried out the 7-days training experiment to assess the performance of the training environment. Scoring and command-consuming are conducted to evaluate the improvement before and after training. The results indicate that the average accuracy is 93.55% and improves from 91.05% in the first stage to 96.05% in the second stage (p = 0.001). Meanwhile, the average score increases from 79.88 in the first session to 96.66 in the last session and tend to be stable (p < 0.001). The average number of commands and collisions to complete the tasks decreases significantly with or without the approximate shortest path (p < 0.001). These results show that the performance of subjects in BCW control achieves improvement and verify the feasibility and effectiveness of the proposed simulated training environment.

Development of a Brain-Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography

Asynchronous brain-computer interfaces (BCIs) based on electroencephalography (EEG) generally suffer from poor performance in terms of classification accuracy and false-positive rate (FPR). Thus, BCI toggle switches based on electrooculogram (EOG) signals were developed to toggle on/off synchronous BCI systems. The conventional BCI toggle switches exhibit fast responses with high accuracy; however, they have a high FPR or cannot be applied to patients with oculomotor impairments. To circumvent these issues, we developed a novel BCI toggle switch that users can employ to toggle on or off synchronous BCIs by holding their breath for a few seconds. Two states-normal breath and breath holding-were classified using a linear discriminant analysis with features extracted from the respiration-modulated photoplethysmography (PPG) signals. A real-time BCI toggle switch was implemented with calibration data trained with only 1-min PPG data. We evaluated the performance of our PPG switch by combining it with a steady-state visual evoked potential-based BCI system that was designed to control four external devices, with regard to the true-positive rate and FPR. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects. All the participants successfully turned on the BCI by holding their breath for approximately 10 s (100% accuracy), and the switch system exhibited a very low FPR of 0.02 false operations per minute, which is the lowest FPR reported thus far. All participants could successfully control external devices in the synchronous BCI mode. Our results demonstrated that the proposed PPG-based BCI toggle switch can be used to implement practical BCIs.

Channel Projection-Based CCA Target Identification Method for an SSVEP-Based BCI System of Quadrotor Helicopter Control

The brain-computer interface (BCI) plays an important role in assisting patients with amyotrophic lateral sclerosis (ALS) to enable them to participate in communication and entertainment. In this study, a novel channel projection-based canonical correlation analysis (CP-CCA) target identification method for steady-state visual evoked potential- (SSVEP-) based BCI system was proposed. The single-channel electroencephalography (EEG) signals of multiple trials were recorded when the subject is under the same stimulus frequency. The CCAs between single-channel EEG signals of multiple trials and sine-cosine reference signals were obtained. Then, the optimal reference signal of each channel was utilized to estimate the test EEG signal. To validate the proposed method, we acquired the training dataset with two testing conditions including the optimal time window length and the number of the trial of training data. The offline experiments conducted a comparison of the proposed method with the traditional canonical correlation analysis (CCA) and power spectrum density analysis (PSDA) method using a 5-class SSVEP dataset that was recorded from 10 subjects. Based on the training dataset, the online 3D-helicopter control experiment was carried out. The offline experimental results showed that the proposed method outperformed the CCA and the PSDA methods in terms of classification accuracy and information transfer rate (ITR). Furthermore, the online experiments of 3-DOF helicopter control achieved an average accuracy of 87.94 ± 5.93% with an ITR of 21.07 ± 4.42 bit/min.

Open-Access fNIRS Dataset for Classification of Unilateral Finger- and Foot-Tapping

Numerous open-access electroencephalography (EEG) datasets have been released and widely employed by EEG researchers. However, not many functional near-infrared spectroscopy (fNIRS) datasets are publicly available. More fNIRS datasets need to be freely accessible in order to facilitate fNIRS studies. Toward this end, we introduce an open-access fNIRS dataset for three-class classification. The concentration changes of oxygenated and reduced hemoglobin were measured, while 30 volunteers repeated each of the three types of overt movements (i.e., left- and right-hand unilateral complex finger-tapping, foot-tapping) for 25 times. The ternary support vector machine (SVM) classification accuracy obtained using leave-one-out cross-validation was estimated at 70.4% ± 18.4% on average. A total of 21 out of 30 volunteers scored a superior binary SVM classification accuracy (left-hand vs. right-hand finger-tapping) of over 80.0%. We believe that the introduced fNIRS dataset can facilitate future fNIRS studies.

An exploration of neural dynamics of motor imagery for people with amyotrophic lateral sclerosis

OBJECTIVE

Studies of the neuropathological effects of amyotrophic lateral sclerosis (ALS) on the underlying motor system have investigated abnormalities in the magnitude and timing of the event-related desynchronization (ERD) and synchronization (ERS) during motor execution (ME). However, the spatio-spectral-temporal dynamics of these sensorimotor oscillations during motor imagery (MI) have not been fully explored for these patients. This study explores the neural dynamics of sensorimotor oscillations for ALS patients during MI by quantifying ERD/ERS features in frequency, time, and space.

APPROACH

Electroencephalogram (EEG) data were recorded from six patients with ALS and 11 age-matched healthy controls (HC) while performing a MI task. ERD/ERS features were extracted using wavelet-based time-frequency analysis and compared between the two groups to quantify the abnormal neural dynamics of ALS in terms of both time and frequency. Topographic correlation analysis was conducted to compare the localization of MI activity between groups and to identify subject-specific frequencies in the µ and β frequency bands.

MAIN RESULTS

Overall, reduced and delayed ERD was observed for ALS patients, particularly during right-hand MI. ERD features were also correlated with ALS clinical scores, specifically disease duration, bulbar, and cognitive functions.

SIGNIFICANCE

The analyses in this study quantify abnormalities in the magnitude and timing of sensorimotor oscillations for ALS patients during MI tasks. Our findings reveal notable differences between MI and existing results on ME in ALS. The observed alterations are speculated to reflect disruptions in the underlying cortical networks involved in MI functions. Quantifying the neural dynamics of MI plays an important role in the study of EEG-based cortical markers for ALS.

Effects of Motor Imagery and Visual Neurofeedback on Activation in the Swallowing Network: A Real-Time fMRI Study

Motor imagery of movements is used as mental strategy in neurofeedback applications to gain voluntary control over activity in motor areas of the brain. In the present functional magnetic resonance imaging (fMRI) study, we first addressed the question whether motor imagery and execution of swallowing activate comparable brain areas, which has been already proven for hand and foot movements. Prior near-infrared spectroscopy (NIRS) studies provide evidence that this is the case in the outer layer of the cortex. With the present fMRI study, we want to expand these prior NIRS findings to the whole brain. Second, we used motor imagery of swallowing as mental strategy during visual neurofeedback to investigate whether one can learn to modulate voluntarily activity in brain regions, which are associated with active swallowing, using real-time fMRI. Eleven healthy adults performed one offline session, in which they executed swallowing movements and imagined swallowing on command during fMRI scanning. Based on this functional localizer task, we identified brain areas active during both tasks and defined individually regions for feedback. During the second session, participants performed two real-time fMRI neurofeedback runs (each run comprised 10 motor imagery trials), in which they should increase voluntarily the activity in the left precentral gyrus by means of motor imagery of swallowing while receiving visual feedback (the visual feedback depicted one's own fMRI signal changes in real-time). Motor execution and imagery of swallowing activated a comparable network of brain areas including the bilateral pre- and postcentral gyrus, inferior frontal gyrus, basal ganglia, insula, SMA, and the cerebellum compared to a resting condition. During neurofeedback training, participants were able to increase the activity in the feedback region (left lateral precentral gyrus) but also in other brain regions, which are generally active during swallowing, compared to the motor imagery offline task. Our results indicate that motor imagery of swallowing is an adequate mental strategy to activate the swallowing network of the whole brain, which might be useful for future treatments of swallowing disorders.

Prediction in Autism by Deep Learning Short-Time Spontaneous Hemodynamic Fluctuations

This study aims to explore the possibility of using a multilayer artificial neural network for the classification between children with autism spectrum disorder (ASD) and typically developing (TD) children based on short-time spontaneous hemodynamic fluctuations. Spontaneous hemodynamic fluctuations were collected by a functional near-infrared spectroscopy setup from bilateral inferior frontal gyrus and temporal cortex in 25 children with ASD and 22 TD children. To perform feature extraction and classification, a multilayer neural network called CGRNN was used which combined a convolution neural network (CNN) and a gate recurrent unit (GRU), since CGRNN has a strong ability in finding characteristic features and acquiring intrinsic relationship in time series. For the training and predicting, short-time (7 s) time-series raw functional near-infrared spectroscopy (fNIRS) signals were used as the input of the network. To avoid the over-fitting problem and effectively extract useful differentiation features from a sample with a very limited size (e.g., 25 ASDs and 22 TDs), a sliding window approach was utilized in which the initially recorded long-time (e.g., 480 s) time-series was divided into many partially overlapped short-time (7 s) sequences. By using this combined deep-learning network, a high accurate classification between ASD and TD could be achieved even with a single optical channel, e.g., 92.2% accuracy, 85.0% sensitivity, and 99.4% specificity. This result implies that the multilayer neural network CGRNN can identify characteristic features associated with ASD even in a short-time spontaneous hemodynamic fluctuation from a single optical channel, and second, the CGRNN can provide highly accurate prediction in ASD.

Multimodal exploration of non-motor neural functions in ALS patients using simultaneous EEG-fNIRS recording

OBJECTIVE

Despite the high prevalence of non-motor impairments reported in patients with amyotrophic lateral sclerosis (ALS), little is known about the functional neural markers underlying such dysfunctions. In this study, a new dual-task multimodal framework relying on simultaneous electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) recordings was developed to characterize integrative non-motor neural functions in people with ALS.

APPROACH

Simultaneous EEG-fNIRS data were recorded from six subjects with ALS and twelve healthy controls. Through a proposed visuo-mental paradigm, subjects performed a set of visuo-mental arithmetic operations. The data recorded were analyzed with respect to event-related changes both in the time and frequency domains for EEG and de/oxygen-hemoglobin level (HbR/HbO) changes for fNIRS. The correlation of EEG spectral features with fNIRS HbO/HbR features were then evaluated to assess the mechanisms of ALS on the electrical (EEG)-vascular (fNIRS) interrelationships.

MAIN RESULTS

We observed overall smaller increases in EEG delta and theta power, decreases in beta power, reductions in HbO responses, and distortions both in early and later EEG event-related potentials in ALS subjects compared to healthy controls. While significant correlations between EEG features and HbO responses were observed in healthy controls, these patterns were absent in ALS patients. Distortions in both electrical and hemodynamic responses are speculated to be associated with cognitive deficits in ALS that center primarily on attentional and working memory processing.

SIGNIFICANCE

Our results highlight the important role of ALS non-motor dysfunctions in electrical and hemodynamic neural dynamics as well as their interrelationships. The insights obtained through this study can enhance our understanding of the underlying non-motor neural processes in ALS and enrich future diagnostic and prognostic techniques.

A Newcomer's Guide to Functional Near Infrared Spectroscopy Experiments

This review presents a practical primer for functional near-infrared spectroscopy (fNIRS) with respect to technology, experimentation, and analysis software. Its purpose is to jump-start interested practitioners considering utilizing a non-invasive, versatile, nevertheless challenging window into the brain using optical methods. We briefly recapitulate relevant anatomical and optical foundations and give a short historical overview. We describe competing types of illumination (trans-illumination, reflectance, and differential reflectance) and data collection methods (continuous wave, time domain and frequency domain). Basic components (light sources, detection, and recording components) of fNIRS systems are presented. Advantages and limitations of fNIRS techniques are offered, followed by a list of very practical recommendations for its use. A variety of experimental and clinical studies with fNIRS are sampled, shedding light on many brain-related ailments. Finally, we describe and discuss a number of freely available analysis and presentation packages suited for data analysis. In conclusion, we recommend fNIRS due to its ever-growing body of clinical applications, state-of-the-art neuroimaging technique and manageable hardware requirements. It can be safely concluded that fNIRS adds a new arrow to the quiver of neuro-medical examinations due to both its great versatility and limited costs.

Mapping cortical network effects of fatigue during a handgrip task by functional near-infrared spectroscopy in physically active and inactive subjects

The temporal evolution of cortical activation and connectivity patterns during a fatiguing handgrip task were studied by functional near-infrared spectroscopy (fNIRS). Twenty-three young adults (18 to 35 years old) were recruited to use a handheld force sensor to perform intermittent handgrip contractions with their dominant hand at their personal maximum voluntary contraction force level for 3.5 s followed by 6.5 s of rest for 120 blocks. Subjects were divided into self-reported physically active and inactive groups, and their hemodynamic activity over the prefrontal and sensory-motor cortices (111 channels) was mapped while they performed this task. Using this fNIRS setup, a more detailed time sequence of cortical activation and connectivity patterns was observed compared to prior studies. A temporal evolution sequence of hemodynamic activation patterns was noted, which was different between the active and the inactive groups. Physically active subjects demonstrated delayed fatigue onset and significantly longer-lasting and more spatially extended functional connectivity (FC) patterns, compared to inactive subjects. The observed differences in activation and FC suggested differences in cortical network adaptation patterns as fatigue set in, which were dependent on subjects' physical activity. The findings of this study suggest that physical activity increases FC with regions involved in motor task control and correlates to extended fatigue onset and enhanced performance.

Fusing Near-Infrared Spectroscopy with Wearable Hemodynamic Measurements Improves Classification of Mental Stress

Human-computer interaction (HCI) technology, and the automatic classification of a person's mental state, are of interest to multiple industries. In this work, the fusion of sensing modalities that monitor the oxygenation of the human prefrontal cortex (PFC) and cardiovascular physiology was evaluated to differentiate between rest, mental arithmetic and N-back memory tasks. A flexible headband to measure near-infrared spectroscopy (NIRS) for quantifying PFC oxygenation, and forehead photoplethysmography (PPG) for assessing peripheral cardiovascular activity was designed. Physiological signals such as the electrocardiogram (ECG) and seismocardiogram (SCG) were collected, along with the measurements obtained using the headband. The setup was tested and validated with a total of 16 human subjects performing a series of arithmetic and N-back memory tasks. Features extracted were related to cardiac and peripheral sympathetic activity, vasomotor tone, pulse wave propagation, and oxygenation. Machine learning techniques were utilized to classify rest, arithmetic, and N-back tasks, using leave-one-subject-out cross validation. Macro-averaged accuracy of 85%, precision of 84%, recall rate of 83%, and F1 score of 80% were obtained from the classification of the three states. Statistical analyses on the subject-based results demonstrate that the fusion of NIRS and peripheral cardiovascular sensing significantly improves the accuracy, precision, recall, and F1 scores, compared to using NIRS sensing alone. Moreover, the fusion significantly improves the precision compared to peripheral cardiovascular sensing alone. The results of this work can be used in the future to design a multi-modal wearable sensing system for classifying mental state for applications such as acute stress detection.

A Graph-Based Hierarchical Attention Model for Movement Intention Detection from EEG Signals

An EEG-based Brain-Computer Interface (BCI) is a system that enables a user to communicate with and intuitively control external devices solely using the user's intentions. Current EEG-based BCI research usually involves a subject-specific adaptation step before a BCI system is ready to be employed by a new user. However, the subject-independent scenario, in which a well-trained model can be directly applied to new users without pre-calibration, is particularly desirable yet rarely explored. Considering this critical gap, our focus in this paper is the subject-independent scenario of EEG-based human intention recognition. We present a G raph-based H ierarchical A ttention M odel (G-HAM) that utilizes the graph structure to represent the spatial information of EEG sensors and the hierarchical attention mechanism to focus on both the most discriminative temporal periods and EEG nodes. Extensive experiments on a large EEG dataset containing 105 subjects indicate that our model is capable of exploiting the underlying invariant EEG patterns across different subjects and generalizing the patterns to new subjects with better performance than a series of state-of-the-art and baseline approaches.

Signal Processing in fNIRS: A Case for the Removal of Systemic Activity for Single Trial Data

Researchers using functional near infrared spectroscopy (fNIRS) are increasingly aware of the problem that conventional filtering methods do not eliminate systemic noise at frequencies overlapping with the task frequency. This is a problem when signals are averaged for analysis, even more so when single trial data are used as in online neurofeedback or BCI applications where insufficiently preprocessed data means feeding back noise instead of brain activity or when looking for brain-behavior relationships on a trial-by-trial basis. For removing this task-related noise statistical approaches have been proposed. Yet as evidence is lacking on how these approaches perform on independent data, choosing one approach over another can be difficult. Here signal quality at the single trial level was considered together with statistical effects to inform this choice. Compared were conventional band-pass filtering and wavelet minimum description length detrending and the combination of both with a more elaborate, published preprocessing approach for a motor execution—motor imagery data set. Temporal consistency between Δ[HbO] and Δ[HbR] and two measures of the spatial specificity of signals that are proposed here served as measures of data quality. Both improved strongly for the combinationed preprocessing approaches. Statistical effects showed a strong tendency toward getting smaller for the combined approaches. This underlines the importance to adequately deal with noise in fNIRS recordings and demonstrates how the quality of statistical correction approaches can be estimated.

Comparison of Feature Vector Compositions to Enhance the Performance of NIRS-BCI-Triggered Robotic Hand Orthosis for Post-Stroke Motor Recovery

Recently, brain–computer interfaces, combined with feedback systems and goal-oriented training, have been investigated for their capacity to promote functional recovery after stroke. Accordingly, we developed a brain–computer interface-triggered robotic hand orthosis that assists hand-closing and hand-opening for post-stroke patients without sufficient motor output. In this system, near-infrared spectroscopy is used to monitor the affected motor cortex, and a linear discriminant analysis-based binary classifier estimates hand posture. The estimated posture then wirelessly triggers the robotic hand orthosis. For better performance of the brain–computer interface, we tested feature windows of different lengths and varying feature vector compositions with motor execution data from seven neurologically intact participants. The interaction between a feature window and a delay in the hemodynamic response significantly affected both classification accuracy (Matthew Correlation Coefficient) and detection latency. The ‘preserving channels’ feature vector was able to increase accuracy by 13.14% and decrease latency by 29.48%, relative to averaging. Oxyhemoglobin combined with deoxyhemoglobin improved accuracy by 3.71% and decreased latency by 6.01% relative to oxyhemoglobin alone. Thus, the best classification performance resulted in an accuracy of 0.7154 and a latency of 2.8515 s. The hand rehabilitation system was successfully implemented using this feature vector composition, which yielded better classification performance.

Evaluation of Neural Degeneration Biomarkers in the Prefrontal Cortex for Early Identification of Patients With Mild Cognitive Impairment: An fNIRS Study

Mild cognitive impairment (MCI), a condition characterizing poor cognition, is associated with aging and depicts early symptoms of severe cognitive impairment, known as Alzheimer's disease (AD). Meanwhile, early detection of MCI can prevent progression to AD. A great deal of research has been performed in the past decade on MCI detection. However, availability of biomarkers for MCI detection requires greater attention. In our study, we evaluated putative and reliable biomarkers for diagnosing MCI by performing different mental tasks (i.e., N-back task, Stroop task, and verbal fluency task) using functional near-infrared spectroscopy (fNIRS) signals on a group of 15 MCI patients and 9 healthy control (HC). The 15 digital biomarkers (i.e., five means, seven slopes, peak, skewness, and kurtosis) and two image biomarkers (t-map, correlation map) in the prefrontal cortex (PFC) (i.e., left PFC, middle PFC, and right PFC) between the MCI and HC groups were investigated by the statistical analysis, linear discriminant analysis (LDA), and convolutional neural network (CNN) individually. The results reveal that the statistical analysis using digital biomarkers (with a p-value < 0.05) could not distinguish the MCI patients from the HC over 60% accuracy. Therefore, the current statistical analysis needs to be improved to be used for diagnosing the MCI patients. The best accuracy with LDA was 76.67% with the N-back and Stroop tasks. However, the CNN classification results trained by image biomarkers showed a high accuracy. In particular, the CNN results trained via t-maps revealed the best accuracy (90.62%) with the N-back task, whereas the CNN result trained by the correlation maps was 85.58% with the N-back task. Also, the results illustrated that investigating the sub-regions (i.e., right, middle, left) of the PFC for detecting MCI would be better than examining the whole PFC. The t-map (or/and the correlation map) is conclusively recommended as an image biomarker for early detection of AD. The combination of CNN and image biomarkers can provide a reliable clinical tool for diagnosing MCI patients.

Effects of Acupuncture Therapy on MCI Patients Using Functional Near-Infrared Spectroscopy

Acupuncture therapy (AT) is a non-pharmacological method of treatment that has been applied to various neurological diseases. However, studies on its longitudinal effect on the neural mechanisms of patients with mild cognitive impairment (MCI) for treatment purposes are still lacking in the literature. In this clinical study, we assess the longitudinal effects of ATs on MCI patients using two methods: (i) Montreal Cognitive Assessment test (MoCA-K, Korean version), and (ii) the hemodynamic response (HR) analyses using functional near-infrared spectroscopy (fNIRS). fNIRS signals of a working memory (WM) task were acquired from the prefrontal cortex. Twelve elderly MCI patients and 12 healthy people were recruited as target and healthy control (HC) groups, respectively. Each group went through an fNIRS scanning procedure three times: The initial data were obtained without any ATs, and subsequently a total of 24 AT sessions were conducted for MCI patients (i.e., MCI-0: the data prior to ATs, MCI-1: after 12 sessions of ATs for 6 weeks, MCI-2: another 12 sessions of ATs for 6 weeks). The mean HR responses of all MCI-0–2 cases were lower than those of HCs. To compare the effects of AT on MCI patients, MoCA-K results, temporal HR data, and spatial activation patterns (i.e., t-maps) were examined. In addition, analyses of functional connectivity (FC) and graph theory upon WM tasks were conducted. With ATs, (i) the averaged MoCA-K test scores were improved (MCI-1, p = 0.002; MCI-2, p = 2.9e^–4); (ii) the mean HR response of WM tasks was increased (p < 0.001); and (iii) the t-maps of MCI-1 and MCI-2 were enhanced. Furthermore, an increased FC in the prefrontal cortex in both MCI-1/MCI-2 cases in comparison to MCI-0 was obtained (p < 0.01), and an increasing trend in the graph theory parameters was observed. All these findings reveal that ATs have a positive impact on improving the cognitive function of MCI patients. In conclusion, ATs can be used as a therapeutic tool for MCI patients as a non-pharmacological method (Clinical trial registration number: KCT 0002451 https://cris.nih.go.kr/cris/en/).

Training -Free Steady-State Visual Evoked Potential Brain-Computer Interface Based on Filter Bank Canonical Correlation Analysis and Spatiotemporal Beamforming Decoding

A brain-computer interface (BCI) provides a novel non-muscular communication pathway for individuals with severe neuromuscular diseases. BCI systems based on steady-state visual evoked potentials (SSVEPs) have high classification accuracy, information transfer rate, and signal-to-noise ratio, giving them high research and application value. However, SSVEP-based BCI has several limitations in real-world applications. The main challenge is how to reduce or eliminate the need for a dedicated training process while maintaining high classification accuracy. Filter bank canonical correlation analysis (FBCCA) is a powerful and widely used feature extraction method for SSVEP-based BCI systems. However, the reference signals of FBCCA are fixed-frequency sine-cosine waves, which makes it difficult to accurately describe the complex, mutative, and individually different physiological SSVEPs. Therefore, there is huge room for improvement in classification performance based on the FBCCA method. In contrast, although spatiotemporal beamforming (BF) detects SSVEPs with high accuracy, it needs an additional training process, which limits its application. In this study, we propose a bimodal decoding algorithm (FBCCA+BF), which combines the advantages of the training-free classification of FBCCA and the data-driven and adaptive features of BF. Six-channel SSVEP data corresponding to eight targets measured from 15 subjects were used to test the effectiveness of three different CCA-based methods, BF, and our proposed FBCCA+BF methods. It was found that the classification accuracies for BF and FBCCA+BF are 95.6% and 92.2%, respectively, which are significantly higher than the other CCA-based methods. Notably, both BF and FBCCA+BF obtain state-of-the-art performance, but FBCCA+BF does this without the need for a dedicated training process. Therefore, we conclude that our proposed FBCCA+BF method provides a training-free and high-accuracy approach for SSVEP-based BCIs.

A Mini-Review on Functional Near-Infrared Spectroscopy (fNIRS): Where Do We Stand, and Where Should We Go?

This mini-review is aimed at briefly summarizing the present status of functional near-infrared spectroscopy (fNIRS) and predicting where the technique should go in the next decade. This mini-review quotes 33 articles on the different fNIRS basics and technical developments and 44 reviews on the fNIRS applications published in the last eight years. The huge number of review articles about a wide spectrum of topics in the field of cognitive and social sciences, functional neuroimaging research, and medicine testifies to the maturity achieved by this non-invasive optical vascular-based functional neuroimaging technique. Today, fNIRS has started to be utilized on healthy subjects while moving freely in different naturalistic settings. Further instrumental developments are expected to be done in the near future to fully satisfy this latter important aspect. In addition, fNIRS procedures, including correction methods for the strong extracranial interferences, need to be standardized before using fNIRS as a clinical tool in individual patients. New research avenues such as interactive neurosciences, cortical activation modulated by different type of sport performance, and cortical activation during neurofeedback training are highlighted.

Use of fNIRS to Characterize the Neural Mechanism of Inter-Individual Rhythmic Movement Coordination

Background: Inter-individual rhythmic movement coordination plays an important role in daily life, particularly in competitive sports. Behaviorally, it is more challenging to coordinate alternating movements than symmetrical movements. The neural activity underlying these different movement coordination modes remains to be clarified, particularly considering complex inter-individual coordination differences.

Methods: To further test the neural basis of inter-individual rhythmic movement coordination, a revised experimental paradigm of inter-individual coordination was adopted. Participants were asked to perform symmetric, alternate, or single movements (swinging the lower part of the leg) in the same rhythm. A multi-channel, continuous wave, functional near-infrared spectral (fNIRS) imaging instrument was used to monitor hemodynamic activity while 40 volunteers (9 male pairs and 11 female pairs) performed the task. Multivariate analyses of variance were conducted to compare mean oxy-hemoglobin concentration ([HbO]) across experimental conditions.

Results: A significant three-way interaction (leg-swing condition × ROI × laterality) on mean [HbO] was observed. Post hoc analysis revealed a significant main effect of leg-swing condition only in brain regions of interest [right inferior parietal lobule (IPL)] contralateral to movement execution. Activation in brain regions of interest [right inferior parietal lobule (IPL)] was much stronger in alternate mode compared with symmetric or single modes, and the differences between symmetric and single mode were not statistically significant. This result suggests that the alternate mode of movement coordination was more likely to be supported by the IPL region than the other modes.

Conclusion: The present findings provide neural evidence relevant to the theory of self-organization of movement coordination, in which an alternating movement mode appeared to be a more demanding condition than symmetrical movement.

Feasibility of a Real-Time Clinical Augmented Reality and Artificial Intelligence Framework for Pain Detection and Localization From the Brain

Background

For many years, clinicians have been seeking for objective pain assessment solutions via neuroimaging techniques, focusing on the brain to detect human pain. Unfortunately, most of those techniques are not applicable in the clinical environment or lack accuracy.

Objective

This study aimed to test the feasibility of a mobile neuroimaging-based clinical augmented reality (AR) and artificial intelligence (AI) framework, CLARAi, for objective pain detection and also localization direct from the patient’s brain in real time.

Methods

Clinical dental pain was triggered in 21 patients by hypersensitive tooth stimulation with 20 consecutive descending cold stimulations (32°C-0°C). We used a portable optical neuroimaging technology, functional near-infrared spectroscopy, to gauge their cortical activity during evoked acute clinical pain. The data were decoded using a neural network (NN)–based AI algorithm to classify hemodynamic response data into pain and no-pain brain states in real time. We tested the performance of several networks (NN with 7 layers, 6 layers, 5 layers, 3 layers, recurrent NN, and long short-term memory network) upon reorganized data features on pain diction and localization in a simulated real-time environment. In addition, we also tested the feasibility of transmitting the neuroimaging data to an AR device, HoloLens, in the same simulated environment, allowing visualization of the ongoing cortical activity on a 3-dimensional brain template virtually plotted on the patients’ head during clinical consult.

Results

The artificial neutral network (3-layer NN) achieved an optimal classification accuracy at 80.37% (126,000/156,680) for pain and no pain discrimination, with positive likelihood ratio (PLR) at 2.35. We further explored a 3-class localization task of left/right side pain and no-pain states, and convolutional NN-6 (6-layer NN) achieved highest classification accuracy at 74.23% (1040/1401) with PLR at 2.02.

Conclusions

Additional studies are needed to optimize and validate our prototype CLARAi framework for other pains and neurologic disorders. However, we presented an innovative and feasible neuroimaging-based AR/AI concept that can potentially transform the human brain into an objective target to visualize and precisely measure and localize pain in real time where it is most needed: in the doctor’s office.

International Registered Report Identifier (IRRID)

RR1-10.2196/13594

Towards Real-Time, Continuous Decoding of Gripping Force From Deep Brain Local Field Potentials

Lack of force information and longevity issues are impediments to the successful translation of brain-computer interface systems for prosthetic control from experimental settings to widespread clinical application. The ability to decode force using deep brain stimulation electrodes in the subthalamic nucleus (STN) of the basal ganglia provides an opportunity to address these limitations. This paper explores the use of various classes of algorithms (Wiener filter, Wiener-Cascade model, Kalman filter, and dynamic neural networks) and recommends the use of a Wiener-Cascade model for decoding force from STN. This recommendation is influenced by a combination of accuracy and practical considerations to enable real-time, continuous operation. This paper demonstrates an ability to decode a continuous signal (force) from the STN in real time, allowing the possibility of decoding more than two states from the brain at low latency.

Intersession Instability in fNIRS-Based Emotion Recognition

Emotion recognition based on neural signals is a promising technique for the detection of patients' emotions for enhancing healthcare. However, emotion-related neural signals, such as from functional near infrared spectroscopy (fNIRS), can be affected by various psychophysiological and environmental factors. There is a paucity of literature regarding data instability and classification instability in fNIRS-based emotion recognition systems, phenomenon which may lead to user dissatisfaction and abandonment. We collected data in an fNIRS-based 2-class emotion recognition test-retest experiment (3 week interval) with visual stimuli emotion induction to examine data instability and its impact on classification accuracy. We found a 22.2% average deterioration of emotion classification accuracy between the two sessions, suggesting that classification instability is a serious problem. We found that the changes in the distributions of the selected neural signal features, as evaluated by Kullback-Leibler (KL) divergence, were a likely cause of the accuracy decline. We analyzed the data instability and our results showed that instability of spatial activation patterns and instability of the hemodynamic response in the most activated region are correlated with accuracy decline. Finally, we propose a method for mitigating classification instability in fNIRS-based emotion recognition based on feature selection for stable features, the first such method to our knowledge. This new feature selection criterion considers not only the separability of features (evaluated by Fisher Score) but also their stability over time (evaluated by KL divergence between feature distributions at different time points). Testing showed that this method led to an approximately 5% improvement in cross-session generalization accuracy.

Functional Spectroscopy Mapping of Pain Processing Cortical Areas During Non-painful Peripheral Electrical Stimulation of the Accessory Spinal Nerve

Peripheral electrical stimulation (PES), which encompasses several techniques with heterogeneous physiological responses, has shown in some cases remarkable outcomes for pain treatment and clinical rehabilitation. However, results are still mixed, mainly because there is a lack of understanding regarding its neural mechanisms of action. In this study, we aimed to assess its effects by measuring cortical activation as indexed by functional near infrared spectroscopy (fNIRS). fNIRS is a functional optical imaging method to evaluate hemodynamic changes in oxygenated (HbO) and de-oxygenated (HbR) blood hemoglobin concentrations in cortical capillary networks that can be related to cortical activity. We hypothesized that non-painful PES of accessory spinal nerve (ASN) can promote cortical activation of sensorimotor cortex (SMC) and dorsolateral prefrontal cortex (DLPFC) pain processing cortical areas. Fifteen healthy volunteers received both active and sham ASN electrical stimulation in a crossover study. The hemodynamic cortical response to unilateral right ASN burst electrical stimulation with 10 Hz was measured by a 40-channel fNIRS system. The effect of ASN electrical stimulation over HbO concentration in cortical areas of interest (CAI) was observed through the activation of right-DLPFC (p = 0.025) and left-SMC (p = 0.042) in the active group but not in sham group. Regarding left-DLPFC (p = 0.610) and right-SMC (p = 0.174) there was no statistical difference between groups. As in non-invasive brain stimulation (NIBS) top-down modulation, bottom-up electrical stimulation to the ASN seems to activate the same critical cortical areas on pain pathways related to sensory-discriminative and affective-motivational pain dimensions. These results provide additional mechanistic evidence to develop and optimize the use of peripheral nerve electrical stimulation as a neuromodulatory tool (NCT 03295370— www.clinicaltrials.gov).

Current State and Future Prospects of EEG and fNIRS in Robot-Assisted Gait Rehabilitation: A Brief Review

Gait and balance impairments are frequently considered as the most significant concerns among individuals suffering from neurological diseases. Robot-assisted gait training (RAGT) has shown to be a promising neurorehabilitation intervention to improve gait recovery in patients following stroke or brain injury by potentially initiating neuroplastic changes. However, the neurophysiological processes underlying gait recovery through RAGT remain poorly understood. As non-invasive, portable neuroimaging techniques, electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) provide new insights regarding the neurophysiological processes occurring during RAGT by measuring different perspectives of brain activity. Due to spatial information about changes in cortical activation patterns and the rapid temporal resolution of bioelectrical changes, more features correlated with brain activation and connectivity can be identified when using fused EEG-fNIRS, thus leading to a detailed understanding of neurophysiological mechanisms underlying motor behavior and impairments due to neurological diseases. Therefore, multi-modal integrations of EEG-fNIRS appear promising for the characterization of neurovascular coupling in brain network dynamics induced by RAGT. In this brief review, we surveyed neuroimaging studies focusing specifically on robotic gait rehabilitation. While previous studies have examined either EEG or fNIRS with respect to RAGT, a multi-modal integration of both approaches is lacking. Based on comparable studies using fused EEG-fNIRS integrations either for guiding non-invasive brain stimulation or as part of brain-machine interface paradigms, the potential of this methodologically combined approach in RAGT is discussed. Future research directions and perspectives for targeted, individualized gait recovery that optimize the outcome and efficiency of RAGT in neurorehabilitation were further derived.

Pre-frontal Cortical Activity During Walking and Turning Is Reliable and Differentiates Across Young, Older Adults and People With Parkinson's Disease

Introduction: Mobility declines with age and further with neurodegenerative disorders, such as Parkinson's disease (PD). Walking and turning ability, in particular, are vital aspects of mobility that deteriorate with age and are further impaired in PD. Such deficits have been linked with reduction in automatic control of movement and the need for compensatory cognitive cortical control via the pre-frontal cortex (PFC), however the underlying neural mechanisms remain unclear. Establishing and using a robust methodology to examine PFC activity during continuous walking and turning via mobile functional near infra-red spectroscopy (fNIRS) may aid in the understanding of mobility deficits and help with development of appropriate therapeutics. This study aimed to: (1) examine test re-test reliability of PFC activity during continuous turning and walking via fNIRS measurement; and (2) compare PFC activity during continuous turning and walking in young, old and Parkinson's subjects. Methods: Twenty-five young (32.3 ± 7.5 years), nineteen older (65.4 ± 7.0 years), and twenty-four PD (69.3 ± 4.1 years) participants performed continuous walking and 360° turning-in-place tasks, each lasting 2 min. Young participants repeated the tasks a second time to allow fNIRS measurement reliability assessment. The primary outcome was PFC activity, assessed via measuring changes in oxygenated hemoglobin (HbO₂) concentrations. Results: PFC activity during continuous walking and turning was moderately reproducible (Intra-class correlation coefficient = 0.67). The PD group had higher PFC activation than young and older adults during walking and turning, with significant group differences for bilateral PFC activation (p = 0.025), left PFC activation (p = 0.012), and the early period (first 40 s) of walking (p = 0.007), with greater activation required in PD. Interestingly, older adults had similar PFC activation to young adults across conditions, however older adults required greater activation than young adults during continuous turning, specifically the early period of the turning task (Cohens d = 0.86). Conclusions: PFC activity can be measured during continuous walking and turning tasks with acceptable reliability, and can differentiate young, older and PD groups. PFC activation was significantly greater in PD compared to young and older adults during walking, particularly when beginning to walk.

Clinical Brain Monitoring with Time Domain NIRS: A Review and Future Perspectives

Near-infrared spectroscopy (NIRS) is an optical technique that can measure brain tissue oxygenation and haemodynamics in real-time and at the patient bedside allowing medical doctors to access important physiological information. However, despite this, the use of NIRS in a clinical environment is hindered due to limitations, such as poor reproducibility, lack of depth sensitivity and poor brain-specificity. Time domain NIRS (or TD-NIRS) can resolve these issues and offer detailed information of the optical properties of the tissue, allowing better physiological information to be retrieved. This is achieved at the cost of increased instrument complexity, operation complexity and price. In this review, we focus on brain monitoring clinical applications of TD-NIRS. A total of 52 publications were identified, spanning the fields of neonatal imaging, stroke assessment, traumatic brain injury (TBI) assessment, brain death assessment, psychiatry, peroperative care, neuronal disorders assessment and communication with patient with locked-in syndrome. In all the publications, the advantages of the TD-NIRS measurement to (1) extract absolute values of haemoglobin concentration and tissue oxygen saturation, (2) assess the reduced scattering coefficient, and (3) separate between extra-cerebral and cerebral tissues, are highlighted; and emphasize the utility of TD-NIRS in a clinical context. In the last sections of this review, we explore the recent developments of TD-NIRS, in terms of instrumentation and methodologies that might impact and broaden its use in the hospital.

fNIRS Evidence for Recognizably Different Positive Emotions

The behavioral differentiation of positive emotions has recently been studied in terms of their discrete adaptive functions or appraising profiles. Some preliminary neurophysiological evidences have been found with electroencephalography or autonomic nervous system measurements such as heart rate, skin conductance, etc. However, the brain's hemodynamic responses to different positive emotions remain largely unknown. In the present study, the functional near-infrared spectroscopy (fNIRS) technique was employed. With this tool, we for the first time reported recognizable discrete positive emotions using fNIRS signals. Thirteen participants watched 30 emotional video clips to elicit 10 typical kinds of positive emotions (joy, gratitude, serenity, interest, hope, pride, amusement, inspiration, awe, and love), and their frontal neural activities were simultaneously recorded with a 24-channel fNIRS system. The multidimensional scaling analysis of participants' subjective ratings on these 10 positive emotions revealed three distinct clusters, which could be interpreted as "playfulness" for amusement, joy, interest, "encouragement" for awe, gratitude, hope, inspiration, pride, and "harmony" for love, serenity. Hemodynamic responses to these three positive emotion clusters showed distinct patterns, and HbO-based individual-level binary classifications between them achieved an averaged accuracy of 73.79 ± 11.49% (77.56 ± 7.39% for encouragement vs. harmony, 73.29 ± 11.87% for playfulness vs. harmony, 70.51 ± 13.96% for encouragement vs. harmony). Benefited from fNIRS's high portability, low running cost and the relative robustness against motion and electrical artifacts, our findings provided support for implementing a more fine-grained emotion recognition system with subdivided positive emotion categories.

Human Brain/Cloud Interface

The Internet comprises a decentralized global system that serves humanity's collective effort to generate, process, and store data, most of which is handled by the rapidly expanding cloud. A stable, secure, real-time system may allow for interfacing the cloud with the human brain. One promising strategy for enabling such a system, denoted here as a "human brain/cloud interface" ("B/CI"), would be based on technologies referred to here as "neuralnanorobotics." Future neuralnanorobotics technologies are anticipated to facilitate accurate diagnoses and eventual cures for the ∼400 conditions that affect the human brain. Neuralnanorobotics may also enable a B/CI with controlled connectivity between neural activity and external data storage and processing, via the direct monitoring of the brain's ∼86 × 109 neurons and ∼2 × 1014 synapses. Subsequent to navigating the human vasculature, three species of neuralnanorobots (endoneurobots, gliabots, and synaptobots) could traverse the blood-brain barrier (BBB), enter the brain parenchyma, ingress into individual human brain cells, and autoposition themselves at the axon initial segments of neurons (endoneurobots), within glial cells (gliabots), and in intimate proximity to synapses (synaptobots). They would then wirelessly transmit up to ∼6 × 1016 bits per second of synaptically processed and encoded human-brain electrical information via auxiliary nanorobotic fiber optics (30 cm3) with the capacity to handle up to 1018 bits/sec and provide rapid data transfer to a cloud based supercomputer for real-time brain-state monitoring and data extraction. A neuralnanorobotically enabled human B/CI might serve as a personalized conduit, allowing persons to obtain direct, instantaneous access to virtually any facet of cumulative human knowledge. Other anticipated applications include myriad opportunities to improve education, intelligence, entertainment, traveling, and other interactive experiences. A specialized application might be the capacity to engage in fully immersive experiential/sensory experiences, including what is referred to here as "transparent shadowing" (TS). Through TS, individuals might experience episodic segments of the lives of other willing participants (locally or remote) to, hopefully, encourage and inspire improved understanding and tolerance among all members of the human family.

Use of a Portable Functional Near-Infrared Spectroscopy (fNIRS) System to Examine Team Experience During Crisis Event Management in Clinical Simulations

Objective: The aim of this study was to investigate the utilization of a portable functional near-infrared spectroscopy (fNIRS) system, the fNIRS Pioneer^TM, to examine team experience in high-fidelity simulation-based crisis event management (CEM) training for anesthesiologists in operating rooms.

Background: Effective evaluation of team performance and experience in CEM simulations is essential for healthcare training and research. Neurophysiological measures with wearable devices can provide useful indicators of team experience to compliment traditional self-report, observer ratings, and behavioral performance measures. fNIRS measured brain blood oxygenation levels and neural synchrony can be used as indicators of workload and team engagement, which is vital for optimal team performance.

Methods: Thirty-three anesthesiologists, who were attending CEM training in two-person teams, participated in this study. The participants varied in their expertise level and the simulation scenarios varied in difficulty level. The oxygenated and de-oxygenated hemoglobin (HbO and HbR) levels in the participants’ prefrontal cortex were derived from data recorded by a portable one-channel fNIRS system worn by all participants throughout CEM training. Team neural synchrony was measured by HbO/HbR wavelet transformation coherence (WTC). Observer-rated workload and self-reported workload and mood were also collected.

Results: At the individual level, the pattern of HbR level corresponded to changes of workload for the individuals in different roles during different phases of a scenario; but this was not the case for HbO level. Thus, HbR level may be a better indicator for individual workload in the studied setting. However, HbR level was insensitive to differences in scenario difficulty and did not correlate with observer-rated or self-reported workload. At the team level, high levels of HbO and HbR WTC were observed during active teamwork. Furthermore, HbO WTC was sensitive to levels of scenario difficulty.

Conclusion: This study showed that it was feasible to use a portable fNIRS system to study workload and team engagement in high-fidelity clinical simulations. However, more work is needed to establish the sensitivity, reliability, and validity of fNIRS measures as indicators of team experience.

BCI Monitor Enhances Electroencephalographic and Cerebral Hemodynamic Activations During Motor Training

Motor imagery-based brain-computer interface (MI-BCI) controlling functional electrical stimulation (FES) is promising for disabled patients to restore their motor functions. However, it remains unclear how much the BCI part can contribute to the functional coupling between the brain and muscle. Specifically, whether it can enhance the cerebral activation for motor training? Here, we investigate the electroencephalographic and cerebral hemodynamic responses for MI-BCI-FES training and MI-FES training, respectively. Twelve healthy subjects were recruited in the motor training study when concurrent electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded. Compared with the MI-FES training conditions, the MI-BCI-FES could induce significantly stronger event-related desynchronization (ERD) and blood oxygen response, which demonstrates that BCI indeed plays a functional role in the closed-loop motor training. Therefore, this paper verifies the feasibility of using BCI to train motor functions in a closed-loop manner.

Classification of Movement Intention Using Independent Components of Premovement EEG

Many previous studies on brain-machine interfaces (BMIs) have focused on electroencephalography (EEG) signals elicited during motor-command execution to generate device commands. However, exploiting pre-execution brain activity related to movement intention could improve the practical applicability of BMIs. Therefore, in this study we investigated whether EEG signals occurring before movement execution could be used to classify movement intention. Six subjects performed reaching tasks that required them to move a cursor to one of four targets distributed horizontally and vertically from the center. Using independent components of EEG acquired during a premovement phase, two-class classifications were performed for left vs. right trials and top vs. bottom trials using a support vector machine. Instructions were presented visually (test) and aurally (condition). In the test condition, accuracy for a single window was about 75%, and it increased to 85% in classification using two windows. In the control condition, accuracy for a single window was about 73%, and it increased to 80% in classification using two windows. Classification results showed that a combination of two windows from different time intervals during the premovement phase improved classification performance in the both conditions compared to a single window classification. By categorizing the independent components according to spatial pattern, we found that information depending on the modality can improve classification performance. We confirmed that EEG signals occurring during movement preparation can be used to control a BMI.

An Information-Theoretic Approach to Quantitative Analysis of the Correspondence Between Skin Blood Flow and Functional Near-Infrared Spectroscopy Measurement in Prefrontal Cortex Activity

Effect of Skin blood flow (SBF) on functional near-infrared spectroscopy (fNIRS) measurement of cortical activity proves to be an illusive subject matter with divided stances in the neuroscientific literature on its extent. Whereas, some reports on its non-significant influence on fNIRS time series of cortical activity, others consider its impact misleading, even detrimental, in analysis of the brain activity as measured by fNIRS. This situation is further escalated by the fact that almost all analytical studies are based on comparison with functional Magnetic Resonance Imaging (fMRI). In this article, we pinpoint the lack of perspective in previous studies on preservation of information content of resulting fNIRS time series once the SBF is attenuated. In doing so, we propose information-theoretic criteria to quantify the necessary and sufficient conditions for SBF attenuation such that the information content of frontal brain activity in resulting fNIRS times series is preserved. We verify these criteria through evaluation of their utility in comparative analysis of principal component (PCA) and independent component (ICA) SBF attenuation algorithms. Our contributions are 2-fold. First, we show that mere reduction of SBF influence on fNIRS time series of frontal activity is insufficient to warrant preservation of cortical activity information. Second, we empirically justify a higher fidelity of PCA-based algorithm in preservation of the fontal activity's information content in comparison with ICA-based approach. Our results suggest that combination of the first two principal components of PCA-based algorithm results in most efficient SBF attenuation while preserving maximum frontal activity's information. These results contribute to the field by presenting a systematic approach to quantification of the SBF as an interfering process during fNIRS measurement, thereby drawing an informed conclusion on this debate. Furthermore, they provide evidence for a reliable choice among existing SBF attenuation algorithms and their inconclusive number of components, thereby ensuring minimum loss of cortical information during SBF attenuation process.

A Systemic Review of Functional Near-Infrared Spectroscopy for Stroke: Current Application and Future Directions

Background: Survivors of stroke often experience significant disability and impaired quality of life. The recovery of motor or cognitive function requires long periods. Neuroimaging could measure changes in the brain and monitor recovery process in order to offer timely treatment and assess the effects of therapy. A non-invasive neuroimaging technique near-infrared spectroscopy (NIRS) with its ambulatory, portable, low-cost nature without fixation of subjects has attracted extensive attention. Methods: We conducted a comprehensive literature review in order to review the use of NIRS in stroke or post-stroke patients in July 2018. NCBI Pubmed database, EMBASE database, Cochrane Library and ScienceDirect database were searched. Results: Overall, we reviewed 66 papers. NIRS has a wide range of application, including in monitoring upper limb, lower limb recovery, motor learning, cortical function recovery, cerebral hemodynamic changes, cerebral oxygenation, as well as in therapeutic method, clinical researches, and evaluation of the risk for stroke. Conclusions: This study provides a preliminary evidence of the application of NIRS in stroke patients as a monitoring, therapeutic, and research tool. Further studies could give more emphasize on the combination of NIRS with other techniques and its utility in the prevention of stroke.

WearLight: Toward a Wearable, Configurable Functional NIR Spectroscopy System for Noninvasive Neuroimaging

Functional near-infrared spectroscopy (fNIRS) has emerged as an effective brain monitoring technique to measure the hemodynamic response of the cortical surface. Its wide popularity and adoption in recent time attribute to its portability, ease of use, and flexibility in multimodal studies involving electroencephalography. While fNIRS is still emerging on various fronts including hardware, software, algorithm, and applications, it still requires overcoming several scientific challenges associated with brain monitoring in naturalistic environments where the human participants are allowed to move and required to perform various tasks stimulating brain behaviors. In response to these challenges and demands, we have developed a wearable fNIRS system, WearLight that was built upon an Internet-of-Things embedded architecture for onboard intelligence, configurability, and data transmission. In addition, we have pursued detailed research and comparative analysis on the design of the optodes encapsulating an near-infrared light source and a detector into 3-D printed material. We performed rigorous experimental studies on human participants to test reliability, signal-to-noise ratio, and configurability. Most importantly, we observed that WearLight has a capacity to measure hemodynamic responses in various setups including arterial occlusion on the forearm and frontal lobe brain activity during breathing exercises in a naturalistic environment. Our promising experimental results provide an evidence of preliminary clinical validation of WearLight. This encourages us to move toward intensive studies involving brain monitoring.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Demonstrating Brain-Level Interactions Between Visuospatial Attentional Demands and Working Memory Load While Driving Using Functional Near-Infrared Spectroscopy

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous 'n' speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2-87.1%; χ2(4) = 19.9, p < 0.001, Kruskal-Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory.

Performance Prediction for a Near-Infrared Spectroscopy-Brain–Computer Interface Using Resting-State Functional Connectivity of the Prefrontal Cortex

One of the most important issues in current brain–computer interface (BCI) research is the prediction of a user’s BCI performance prior to the main BCI session because it would be useful to reduce the time required to determine the BCI paradigm best suited to that user. In electroencephalography (EEG)-BCI research, whether a user has low BCI performance toward a specific BCI paradigm has been estimated using a variety of resting-state EEG features. However, no previous study has attempted to predict the performance of near-infrared spectroscopy (NIRS)-BCI using resting-state NIRS data recorded before the main BCI experiment. In this study, we investigated whether the performance of an NIRS-BCI discriminating a mental arithmetic task from the baseline state could be predicted using resting-state functional connectivity (RSFC) of the prefrontal cortex. The investigation of NIRS signals recorded from 29 participants revealed that the RSFC between bilateral channels in the prefrontal area was negatively correlated with subsequent BCI performance (e.g. a fitted line for the RSFC between L2 and R2 channels explains 41% of BCI performance variation). We expect that our indicator can be used to predict BCI performance of an individual user prior to the main NIRS-BCI experiments, thereby facilitating implementation of more efficient NIRS-BCI systems.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

Applications of Functional Near-Infrared Spectroscopy (fNIRS) Neuroimaging in Exercise⁻Cognition Science: A Systematic, Methodology-Focused Review

For cognitive processes to function well, it is essential that the brain is optimally supplied with oxygen and blood. In recent years, evidence has emerged suggesting that cerebral oxygenation and hemodynamics can be modified with physical activity. To better understand the relationship between cerebral oxygenation/hemodynamics, physical activity, and cognition, the application of state-of-the art neuroimaging tools is essential. Functional near-infrared spectroscopy (fNIRS) is such a neuroimaging tool especially suitable to investigate the effects of physical activity/exercises on cerebral oxygenation and hemodynamics due to its capability to quantify changes in the concentration of oxygenated hemoglobin (oxyHb) and deoxygenated hemoglobin (deoxyHb) non-invasively in the human brain. However, currently there is no clear standardized procedure regarding the application, data processing, and data analysis of fNIRS, and there is a large heterogeneity regarding how fNIRS is applied in the field of exercise⁻cognition science. Therefore, this review aims to summarize the current methodological knowledge about fNIRS application in studies measuring the cortical hemodynamic responses during cognitive testing (i) prior and after different physical activities interventions, and (ii) in cross-sectional studies accounting for the physical fitness level of their participants. Based on the review of the methodology of 35 as relevant considered publications, we outline recommendations for future fNIRS studies in the field of exercise⁻cognition science.

A Modified Common Spatial Pattern Algorithm Customized for Feature Dimensionality Reduction in fNIRS-Based BCIs

Functional near-infrared spectroscopy (fNIRS) is a non-invasive multi-channel imaging tool for assessing brain activities, which has shown its high potential in brain-computer interface (BCI) technique. Most previous studies have focused on constructing high dimensional features from whole channels, adding to the complexity of their classifiers. Another multi-channel source for BCI is electroencephalograph (EEG), which possesses different spatial and temporal features from fNIRS. In EEG field, Common Spatial Pattern (CSP) algorithm is widely used aimed at dimensionality reduction. In our article, we modified it based on the characteristics of fNIRS and evaluated its effectiveness in discriminating Mental Arithmetic (MA) against resting status in an open-access dataset. The Modified Common Spatial Pattern algorithm significantly outperforms CSP algorithm in fNIRS-based BCI and shows its potential in further BCI related explorations.

Within- and Between-Session Prefrontal Cortex Response to Virtual Reality Exposure Therapy for Acrophobia

Exposure Therapy (ET) has demonstrated its efficacy in the treatment of phobias, anxiety and Post-traumatic Stress Disorder (PTSD), however, it suffers a high drop-out rate because of too low or too high patient engagement in treatment. Virtual Reality Exposure Therapy (VRET) is comparably effective regarding symptom reduction and offers an alternative tool to facilitate engagement for avoidant participants. Neuroimaging studies have demonstrated that both ET and VRET normalize brain activity within a fear circuit. However, previous studies have employed brain imaging technology which restricts people's movement and hides their body, surroundings and therapist from view. This is at odds with the way engagement is typically controlled. We used a novel combination of neural imaging and VR technology-Functional Near-Infrared Spectroscopy (fNIRS) and Immersive Projection Technology (IPT), to avoid these limitations. Although there are a few studies that have investigated the effect of VRET on a brain function after the treatment, the present study utilized technologies which promote ecological validity to measure brain changes after VRET treatment. Furthermore, there are no studies that have measured brain activity within VRET session. In this study brain activity within the prefrontal cortex (PFC) was measured during three consecutive exposure sessions. N = 13 acrophobic volunteers were asked to walk on a virtual plank with a 6 m drop below. Changes in oxygenated (HbO) hemoglobin concentrations in the PFC were measured in three blocks using fNIRS. Consistent with previous functional magnetic resonance imaging (fMRI) studies, the analysis showed decreased activity in the DLPFC and MPFC during first exposure. The activity increased toward normal across three sessions. The study demonstrates potential efficacy of a method for measuring within-session neural response to virtual stimuli that could be replicated within clinics and research institutes, with equipment better suited to an ET session and at fraction of the cost, when compared to fMRI. This has application in widening access to, and increasing ecological validity of, immersive neuroimaging across understanding, diagnosis, assessment and treatment of, a range of mental disorders such as phobia, anxiety and PTSD or addictions.