fNIRS-based brain-computer interfaces: a review

Towards Real-Time, Continuous Decoding of Gripping Force From Deep Brain Local Field Potentials

Lack of force information and longevity issues are impediments to the successful translation of brain-computer interface systems for prosthetic control from experimental settings to widespread clinical application. The ability to decode force using deep brain stimulation electrodes in the subthalamic nucleus (STN) of the basal ganglia provides an opportunity to address these limitations. This paper explores the use of various classes of algorithms (Wiener filter, Wiener-Cascade model, Kalman filter, and dynamic neural networks) and recommends the use of a Wiener-Cascade model for decoding force from STN. This recommendation is influenced by a combination of accuracy and practical considerations to enable real-time, continuous operation. This paper demonstrates an ability to decode a continuous signal (force) from the STN in real time, allowing the possibility of decoding more than two states from the brain at low latency.

Intersession Instability in fNIRS-Based Emotion Recognition

Emotion recognition based on neural signals is a promising technique for the detection of patients' emotions for enhancing healthcare. However, emotion-related neural signals, such as from functional near infrared spectroscopy (fNIRS), can be affected by various psychophysiological and environmental factors. There is a paucity of literature regarding data instability and classification instability in fNIRS-based emotion recognition systems, phenomenon which may lead to user dissatisfaction and abandonment. We collected data in an fNIRS-based 2-class emotion recognition test-retest experiment (3 week interval) with visual stimuli emotion induction to examine data instability and its impact on classification accuracy. We found a 22.2% average deterioration of emotion classification accuracy between the two sessions, suggesting that classification instability is a serious problem. We found that the changes in the distributions of the selected neural signal features, as evaluated by Kullback-Leibler (KL) divergence, were a likely cause of the accuracy decline. We analyzed the data instability and our results showed that instability of spatial activation patterns and instability of the hemodynamic response in the most activated region are correlated with accuracy decline. Finally, we propose a method for mitigating classification instability in fNIRS-based emotion recognition based on feature selection for stable features, the first such method to our knowledge. This new feature selection criterion considers not only the separability of features (evaluated by Fisher Score) but also their stability over time (evaluated by KL divergence between feature distributions at different time points). Testing showed that this method led to an approximately 5% improvement in cross-session generalization accuracy.

Functional Spectroscopy Mapping of Pain Processing Cortical Areas During Non-painful Peripheral Electrical Stimulation of the Accessory Spinal Nerve

Peripheral electrical stimulation (PES), which encompasses several techniques with heterogeneous physiological responses, has shown in some cases remarkable outcomes for pain treatment and clinical rehabilitation. However, results are still mixed, mainly because there is a lack of understanding regarding its neural mechanisms of action. In this study, we aimed to assess its effects by measuring cortical activation as indexed by functional near infrared spectroscopy (fNIRS). fNIRS is a functional optical imaging method to evaluate hemodynamic changes in oxygenated (HbO) and de-oxygenated (HbR) blood hemoglobin concentrations in cortical capillary networks that can be related to cortical activity. We hypothesized that non-painful PES of accessory spinal nerve (ASN) can promote cortical activation of sensorimotor cortex (SMC) and dorsolateral prefrontal cortex (DLPFC) pain processing cortical areas. Fifteen healthy volunteers received both active and sham ASN electrical stimulation in a crossover study. The hemodynamic cortical response to unilateral right ASN burst electrical stimulation with 10 Hz was measured by a 40-channel fNIRS system. The effect of ASN electrical stimulation over HbO concentration in cortical areas of interest (CAI) was observed through the activation of right-DLPFC (p = 0.025) and left-SMC (p = 0.042) in the active group but not in sham group. Regarding left-DLPFC (p = 0.610) and right-SMC (p = 0.174) there was no statistical difference between groups. As in non-invasive brain stimulation (NIBS) top-down modulation, bottom-up electrical stimulation to the ASN seems to activate the same critical cortical areas on pain pathways related to sensory-discriminative and affective-motivational pain dimensions. These results provide additional mechanistic evidence to develop and optimize the use of peripheral nerve electrical stimulation as a neuromodulatory tool (NCT 03295370— www.clinicaltrials.gov).

Current State and Future Prospects of EEG and fNIRS in Robot-Assisted Gait Rehabilitation: A Brief Review

Gait and balance impairments are frequently considered as the most significant concerns among individuals suffering from neurological diseases. Robot-assisted gait training (RAGT) has shown to be a promising neurorehabilitation intervention to improve gait recovery in patients following stroke or brain injury by potentially initiating neuroplastic changes. However, the neurophysiological processes underlying gait recovery through RAGT remain poorly understood. As non-invasive, portable neuroimaging techniques, electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) provide new insights regarding the neurophysiological processes occurring during RAGT by measuring different perspectives of brain activity. Due to spatial information about changes in cortical activation patterns and the rapid temporal resolution of bioelectrical changes, more features correlated with brain activation and connectivity can be identified when using fused EEG-fNIRS, thus leading to a detailed understanding of neurophysiological mechanisms underlying motor behavior and impairments due to neurological diseases. Therefore, multi-modal integrations of EEG-fNIRS appear promising for the characterization of neurovascular coupling in brain network dynamics induced by RAGT. In this brief review, we surveyed neuroimaging studies focusing specifically on robotic gait rehabilitation. While previous studies have examined either EEG or fNIRS with respect to RAGT, a multi-modal integration of both approaches is lacking. Based on comparable studies using fused EEG-fNIRS integrations either for guiding non-invasive brain stimulation or as part of brain-machine interface paradigms, the potential of this methodologically combined approach in RAGT is discussed. Future research directions and perspectives for targeted, individualized gait recovery that optimize the outcome and efficiency of RAGT in neurorehabilitation were further derived.

Pre-frontal Cortical Activity During Walking and Turning Is Reliable and Differentiates Across Young, Older Adults and People With Parkinson's Disease

Introduction: Mobility declines with age and further with neurodegenerative disorders, such as Parkinson's disease (PD). Walking and turning ability, in particular, are vital aspects of mobility that deteriorate with age and are further impaired in PD. Such deficits have been linked with reduction in automatic control of movement and the need for compensatory cognitive cortical control via the pre-frontal cortex (PFC), however the underlying neural mechanisms remain unclear. Establishing and using a robust methodology to examine PFC activity during continuous walking and turning via mobile functional near infra-red spectroscopy (fNIRS) may aid in the understanding of mobility deficits and help with development of appropriate therapeutics. This study aimed to: (1) examine test re-test reliability of PFC activity during continuous turning and walking via fNIRS measurement; and (2) compare PFC activity during continuous turning and walking in young, old and Parkinson's subjects. Methods: Twenty-five young (32.3 ± 7.5 years), nineteen older (65.4 ± 7.0 years), and twenty-four PD (69.3 ± 4.1 years) participants performed continuous walking and 360° turning-in-place tasks, each lasting 2 min. Young participants repeated the tasks a second time to allow fNIRS measurement reliability assessment. The primary outcome was PFC activity, assessed via measuring changes in oxygenated hemoglobin (HbO₂) concentrations. Results: PFC activity during continuous walking and turning was moderately reproducible (Intra-class correlation coefficient = 0.67). The PD group had higher PFC activation than young and older adults during walking and turning, with significant group differences for bilateral PFC activation (p = 0.025), left PFC activation (p = 0.012), and the early period (first 40 s) of walking (p = 0.007), with greater activation required in PD. Interestingly, older adults had similar PFC activation to young adults across conditions, however older adults required greater activation than young adults during continuous turning, specifically the early period of the turning task (Cohens d = 0.86). Conclusions: PFC activity can be measured during continuous walking and turning tasks with acceptable reliability, and can differentiate young, older and PD groups. PFC activation was significantly greater in PD compared to young and older adults during walking, particularly when beginning to walk.

Clinical Brain Monitoring with Time Domain NIRS: A Review and Future Perspectives

Near-infrared spectroscopy (NIRS) is an optical technique that can measure brain tissue oxygenation and haemodynamics in real-time and at the patient bedside allowing medical doctors to access important physiological information. However, despite this, the use of NIRS in a clinical environment is hindered due to limitations, such as poor reproducibility, lack of depth sensitivity and poor brain-specificity. Time domain NIRS (or TD-NIRS) can resolve these issues and offer detailed information of the optical properties of the tissue, allowing better physiological information to be retrieved. This is achieved at the cost of increased instrument complexity, operation complexity and price. In this review, we focus on brain monitoring clinical applications of TD-NIRS. A total of 52 publications were identified, spanning the fields of neonatal imaging, stroke assessment, traumatic brain injury (TBI) assessment, brain death assessment, psychiatry, peroperative care, neuronal disorders assessment and communication with patient with locked-in syndrome. In all the publications, the advantages of the TD-NIRS measurement to (1) extract absolute values of haemoglobin concentration and tissue oxygen saturation, (2) assess the reduced scattering coefficient, and (3) separate between extra-cerebral and cerebral tissues, are highlighted; and emphasize the utility of TD-NIRS in a clinical context. In the last sections of this review, we explore the recent developments of TD-NIRS, in terms of instrumentation and methodologies that might impact and broaden its use in the hospital.

fNIRS Evidence for Recognizably Different Positive Emotions

The behavioral differentiation of positive emotions has recently been studied in terms of their discrete adaptive functions or appraising profiles. Some preliminary neurophysiological evidences have been found with electroencephalography or autonomic nervous system measurements such as heart rate, skin conductance, etc. However, the brain's hemodynamic responses to different positive emotions remain largely unknown. In the present study, the functional near-infrared spectroscopy (fNIRS) technique was employed. With this tool, we for the first time reported recognizable discrete positive emotions using fNIRS signals. Thirteen participants watched 30 emotional video clips to elicit 10 typical kinds of positive emotions (joy, gratitude, serenity, interest, hope, pride, amusement, inspiration, awe, and love), and their frontal neural activities were simultaneously recorded with a 24-channel fNIRS system. The multidimensional scaling analysis of participants' subjective ratings on these 10 positive emotions revealed three distinct clusters, which could be interpreted as "playfulness" for amusement, joy, interest, "encouragement" for awe, gratitude, hope, inspiration, pride, and "harmony" for love, serenity. Hemodynamic responses to these three positive emotion clusters showed distinct patterns, and HbO-based individual-level binary classifications between them achieved an averaged accuracy of 73.79 ± 11.49% (77.56 ± 7.39% for encouragement vs. harmony, 73.29 ± 11.87% for playfulness vs. harmony, 70.51 ± 13.96% for encouragement vs. harmony). Benefited from fNIRS's high portability, low running cost and the relative robustness against motion and electrical artifacts, our findings provided support for implementing a more fine-grained emotion recognition system with subdivided positive emotion categories.

Human Brain/Cloud Interface

The Internet comprises a decentralized global system that serves humanity's collective effort to generate, process, and store data, most of which is handled by the rapidly expanding cloud. A stable, secure, real-time system may allow for interfacing the cloud with the human brain. One promising strategy for enabling such a system, denoted here as a "human brain/cloud interface" ("B/CI"), would be based on technologies referred to here as "neuralnanorobotics." Future neuralnanorobotics technologies are anticipated to facilitate accurate diagnoses and eventual cures for the ∼400 conditions that affect the human brain. Neuralnanorobotics may also enable a B/CI with controlled connectivity between neural activity and external data storage and processing, via the direct monitoring of the brain's ∼86 × 109 neurons and ∼2 × 1014 synapses. Subsequent to navigating the human vasculature, three species of neuralnanorobots (endoneurobots, gliabots, and synaptobots) could traverse the blood-brain barrier (BBB), enter the brain parenchyma, ingress into individual human brain cells, and autoposition themselves at the axon initial segments of neurons (endoneurobots), within glial cells (gliabots), and in intimate proximity to synapses (synaptobots). They would then wirelessly transmit up to ∼6 × 1016 bits per second of synaptically processed and encoded human-brain electrical information via auxiliary nanorobotic fiber optics (30 cm3) with the capacity to handle up to 1018 bits/sec and provide rapid data transfer to a cloud based supercomputer for real-time brain-state monitoring and data extraction. A neuralnanorobotically enabled human B/CI might serve as a personalized conduit, allowing persons to obtain direct, instantaneous access to virtually any facet of cumulative human knowledge. Other anticipated applications include myriad opportunities to improve education, intelligence, entertainment, traveling, and other interactive experiences. A specialized application might be the capacity to engage in fully immersive experiential/sensory experiences, including what is referred to here as "transparent shadowing" (TS). Through TS, individuals might experience episodic segments of the lives of other willing participants (locally or remote) to, hopefully, encourage and inspire improved understanding and tolerance among all members of the human family.

Use of a Portable Functional Near-Infrared Spectroscopy (fNIRS) System to Examine Team Experience During Crisis Event Management in Clinical Simulations

Objective: The aim of this study was to investigate the utilization of a portable functional near-infrared spectroscopy (fNIRS) system, the fNIRS Pioneer^TM, to examine team experience in high-fidelity simulation-based crisis event management (CEM) training for anesthesiologists in operating rooms.

Background: Effective evaluation of team performance and experience in CEM simulations is essential for healthcare training and research. Neurophysiological measures with wearable devices can provide useful indicators of team experience to compliment traditional self-report, observer ratings, and behavioral performance measures. fNIRS measured brain blood oxygenation levels and neural synchrony can be used as indicators of workload and team engagement, which is vital for optimal team performance.

Methods: Thirty-three anesthesiologists, who were attending CEM training in two-person teams, participated in this study. The participants varied in their expertise level and the simulation scenarios varied in difficulty level. The oxygenated and de-oxygenated hemoglobin (HbO and HbR) levels in the participants’ prefrontal cortex were derived from data recorded by a portable one-channel fNIRS system worn by all participants throughout CEM training. Team neural synchrony was measured by HbO/HbR wavelet transformation coherence (WTC). Observer-rated workload and self-reported workload and mood were also collected.

Results: At the individual level, the pattern of HbR level corresponded to changes of workload for the individuals in different roles during different phases of a scenario; but this was not the case for HbO level. Thus, HbR level may be a better indicator for individual workload in the studied setting. However, HbR level was insensitive to differences in scenario difficulty and did not correlate with observer-rated or self-reported workload. At the team level, high levels of HbO and HbR WTC were observed during active teamwork. Furthermore, HbO WTC was sensitive to levels of scenario difficulty.

Conclusion: This study showed that it was feasible to use a portable fNIRS system to study workload and team engagement in high-fidelity clinical simulations. However, more work is needed to establish the sensitivity, reliability, and validity of fNIRS measures as indicators of team experience.

BCI Monitor Enhances Electroencephalographic and Cerebral Hemodynamic Activations During Motor Training

Motor imagery-based brain-computer interface (MI-BCI) controlling functional electrical stimulation (FES) is promising for disabled patients to restore their motor functions. However, it remains unclear how much the BCI part can contribute to the functional coupling between the brain and muscle. Specifically, whether it can enhance the cerebral activation for motor training? Here, we investigate the electroencephalographic and cerebral hemodynamic responses for MI-BCI-FES training and MI-FES training, respectively. Twelve healthy subjects were recruited in the motor training study when concurrent electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded. Compared with the MI-FES training conditions, the MI-BCI-FES could induce significantly stronger event-related desynchronization (ERD) and blood oxygen response, which demonstrates that BCI indeed plays a functional role in the closed-loop motor training. Therefore, this paper verifies the feasibility of using BCI to train motor functions in a closed-loop manner.

Classification of Movement Intention Using Independent Components of Premovement EEG

Many previous studies on brain-machine interfaces (BMIs) have focused on electroencephalography (EEG) signals elicited during motor-command execution to generate device commands. However, exploiting pre-execution brain activity related to movement intention could improve the practical applicability of BMIs. Therefore, in this study we investigated whether EEG signals occurring before movement execution could be used to classify movement intention. Six subjects performed reaching tasks that required them to move a cursor to one of four targets distributed horizontally and vertically from the center. Using independent components of EEG acquired during a premovement phase, two-class classifications were performed for left vs. right trials and top vs. bottom trials using a support vector machine. Instructions were presented visually (test) and aurally (condition). In the test condition, accuracy for a single window was about 75%, and it increased to 85% in classification using two windows. In the control condition, accuracy for a single window was about 73%, and it increased to 80% in classification using two windows. Classification results showed that a combination of two windows from different time intervals during the premovement phase improved classification performance in the both conditions compared to a single window classification. By categorizing the independent components according to spatial pattern, we found that information depending on the modality can improve classification performance. We confirmed that EEG signals occurring during movement preparation can be used to control a BMI.

An Information-Theoretic Approach to Quantitative Analysis of the Correspondence Between Skin Blood Flow and Functional Near-Infrared Spectroscopy Measurement in Prefrontal Cortex Activity

Effect of Skin blood flow (SBF) on functional near-infrared spectroscopy (fNIRS) measurement of cortical activity proves to be an illusive subject matter with divided stances in the neuroscientific literature on its extent. Whereas, some reports on its non-significant influence on fNIRS time series of cortical activity, others consider its impact misleading, even detrimental, in analysis of the brain activity as measured by fNIRS. This situation is further escalated by the fact that almost all analytical studies are based on comparison with functional Magnetic Resonance Imaging (fMRI). In this article, we pinpoint the lack of perspective in previous studies on preservation of information content of resulting fNIRS time series once the SBF is attenuated. In doing so, we propose information-theoretic criteria to quantify the necessary and sufficient conditions for SBF attenuation such that the information content of frontal brain activity in resulting fNIRS times series is preserved. We verify these criteria through evaluation of their utility in comparative analysis of principal component (PCA) and independent component (ICA) SBF attenuation algorithms. Our contributions are 2-fold. First, we show that mere reduction of SBF influence on fNIRS time series of frontal activity is insufficient to warrant preservation of cortical activity information. Second, we empirically justify a higher fidelity of PCA-based algorithm in preservation of the fontal activity's information content in comparison with ICA-based approach. Our results suggest that combination of the first two principal components of PCA-based algorithm results in most efficient SBF attenuation while preserving maximum frontal activity's information. These results contribute to the field by presenting a systematic approach to quantification of the SBF as an interfering process during fNIRS measurement, thereby drawing an informed conclusion on this debate. Furthermore, they provide evidence for a reliable choice among existing SBF attenuation algorithms and their inconclusive number of components, thereby ensuring minimum loss of cortical information during SBF attenuation process.

A Systemic Review of Functional Near-Infrared Spectroscopy for Stroke: Current Application and Future Directions

Background: Survivors of stroke often experience significant disability and impaired quality of life. The recovery of motor or cognitive function requires long periods. Neuroimaging could measure changes in the brain and monitor recovery process in order to offer timely treatment and assess the effects of therapy. A non-invasive neuroimaging technique near-infrared spectroscopy (NIRS) with its ambulatory, portable, low-cost nature without fixation of subjects has attracted extensive attention. Methods: We conducted a comprehensive literature review in order to review the use of NIRS in stroke or post-stroke patients in July 2018. NCBI Pubmed database, EMBASE database, Cochrane Library and ScienceDirect database were searched. Results: Overall, we reviewed 66 papers. NIRS has a wide range of application, including in monitoring upper limb, lower limb recovery, motor learning, cortical function recovery, cerebral hemodynamic changes, cerebral oxygenation, as well as in therapeutic method, clinical researches, and evaluation of the risk for stroke. Conclusions: This study provides a preliminary evidence of the application of NIRS in stroke patients as a monitoring, therapeutic, and research tool. Further studies could give more emphasize on the combination of NIRS with other techniques and its utility in the prevention of stroke.

WearLight: Toward a Wearable, Configurable Functional NIR Spectroscopy System for Noninvasive Neuroimaging

Functional near-infrared spectroscopy (fNIRS) has emerged as an effective brain monitoring technique to measure the hemodynamic response of the cortical surface. Its wide popularity and adoption in recent time attribute to its portability, ease of use, and flexibility in multimodal studies involving electroencephalography. While fNIRS is still emerging on various fronts including hardware, software, algorithm, and applications, it still requires overcoming several scientific challenges associated with brain monitoring in naturalistic environments where the human participants are allowed to move and required to perform various tasks stimulating brain behaviors. In response to these challenges and demands, we have developed a wearable fNIRS system, WearLight that was built upon an Internet-of-Things embedded architecture for onboard intelligence, configurability, and data transmission. In addition, we have pursued detailed research and comparative analysis on the design of the optodes encapsulating an near-infrared light source and a detector into 3-D printed material. We performed rigorous experimental studies on human participants to test reliability, signal-to-noise ratio, and configurability. Most importantly, we observed that WearLight has a capacity to measure hemodynamic responses in various setups including arterial occlusion on the forearm and frontal lobe brain activity during breathing exercises in a naturalistic environment. Our promising experimental results provide an evidence of preliminary clinical validation of WearLight. This encourages us to move toward intensive studies involving brain monitoring.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Demonstrating Brain-Level Interactions Between Visuospatial Attentional Demands and Working Memory Load While Driving Using Functional Near-Infrared Spectroscopy

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous 'n' speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2-87.1%; χ2(4) = 19.9, p < 0.001, Kruskal-Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory.

Performance Prediction for a Near-Infrared Spectroscopy-Brain–Computer Interface Using Resting-State Functional Connectivity of the Prefrontal Cortex

One of the most important issues in current brain–computer interface (BCI) research is the prediction of a user’s BCI performance prior to the main BCI session because it would be useful to reduce the time required to determine the BCI paradigm best suited to that user. In electroencephalography (EEG)-BCI research, whether a user has low BCI performance toward a specific BCI paradigm has been estimated using a variety of resting-state EEG features. However, no previous study has attempted to predict the performance of near-infrared spectroscopy (NIRS)-BCI using resting-state NIRS data recorded before the main BCI experiment. In this study, we investigated whether the performance of an NIRS-BCI discriminating a mental arithmetic task from the baseline state could be predicted using resting-state functional connectivity (RSFC) of the prefrontal cortex. The investigation of NIRS signals recorded from 29 participants revealed that the RSFC between bilateral channels in the prefrontal area was negatively correlated with subsequent BCI performance (e.g. a fitted line for the RSFC between L2 and R2 channels explains 41% of BCI performance variation). We expect that our indicator can be used to predict BCI performance of an individual user prior to the main NIRS-BCI experiments, thereby facilitating implementation of more efficient NIRS-BCI systems.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

Applications of Functional Near-Infrared Spectroscopy (fNIRS) Neuroimaging in Exercise⁻Cognition Science: A Systematic, Methodology-Focused Review

For cognitive processes to function well, it is essential that the brain is optimally supplied with oxygen and blood. In recent years, evidence has emerged suggesting that cerebral oxygenation and hemodynamics can be modified with physical activity. To better understand the relationship between cerebral oxygenation/hemodynamics, physical activity, and cognition, the application of state-of-the art neuroimaging tools is essential. Functional near-infrared spectroscopy (fNIRS) is such a neuroimaging tool especially suitable to investigate the effects of physical activity/exercises on cerebral oxygenation and hemodynamics due to its capability to quantify changes in the concentration of oxygenated hemoglobin (oxyHb) and deoxygenated hemoglobin (deoxyHb) non-invasively in the human brain. However, currently there is no clear standardized procedure regarding the application, data processing, and data analysis of fNIRS, and there is a large heterogeneity regarding how fNIRS is applied in the field of exercise⁻cognition science. Therefore, this review aims to summarize the current methodological knowledge about fNIRS application in studies measuring the cortical hemodynamic responses during cognitive testing (i) prior and after different physical activities interventions, and (ii) in cross-sectional studies accounting for the physical fitness level of their participants. Based on the review of the methodology of 35 as relevant considered publications, we outline recommendations for future fNIRS studies in the field of exercise⁻cognition science.

A Modified Common Spatial Pattern Algorithm Customized for Feature Dimensionality Reduction in fNIRS-Based BCIs

Functional near-infrared spectroscopy (fNIRS) is a non-invasive multi-channel imaging tool for assessing brain activities, which has shown its high potential in brain-computer interface (BCI) technique. Most previous studies have focused on constructing high dimensional features from whole channels, adding to the complexity of their classifiers. Another multi-channel source for BCI is electroencephalograph (EEG), which possesses different spatial and temporal features from fNIRS. In EEG field, Common Spatial Pattern (CSP) algorithm is widely used aimed at dimensionality reduction. In our article, we modified it based on the characteristics of fNIRS and evaluated its effectiveness in discriminating Mental Arithmetic (MA) against resting status in an open-access dataset. The Modified Common Spatial Pattern algorithm significantly outperforms CSP algorithm in fNIRS-based BCI and shows its potential in further BCI related explorations.

Within- and Between-Session Prefrontal Cortex Response to Virtual Reality Exposure Therapy for Acrophobia

Exposure Therapy (ET) has demonstrated its efficacy in the treatment of phobias, anxiety and Post-traumatic Stress Disorder (PTSD), however, it suffers a high drop-out rate because of too low or too high patient engagement in treatment. Virtual Reality Exposure Therapy (VRET) is comparably effective regarding symptom reduction and offers an alternative tool to facilitate engagement for avoidant participants. Neuroimaging studies have demonstrated that both ET and VRET normalize brain activity within a fear circuit. However, previous studies have employed brain imaging technology which restricts people's movement and hides their body, surroundings and therapist from view. This is at odds with the way engagement is typically controlled. We used a novel combination of neural imaging and VR technology-Functional Near-Infrared Spectroscopy (fNIRS) and Immersive Projection Technology (IPT), to avoid these limitations. Although there are a few studies that have investigated the effect of VRET on a brain function after the treatment, the present study utilized technologies which promote ecological validity to measure brain changes after VRET treatment. Furthermore, there are no studies that have measured brain activity within VRET session. In this study brain activity within the prefrontal cortex (PFC) was measured during three consecutive exposure sessions. N = 13 acrophobic volunteers were asked to walk on a virtual plank with a 6 m drop below. Changes in oxygenated (HbO) hemoglobin concentrations in the PFC were measured in three blocks using fNIRS. Consistent with previous functional magnetic resonance imaging (fMRI) studies, the analysis showed decreased activity in the DLPFC and MPFC during first exposure. The activity increased toward normal across three sessions. The study demonstrates potential efficacy of a method for measuring within-session neural response to virtual stimuli that could be replicated within clinics and research institutes, with equipment better suited to an ET session and at fraction of the cost, when compared to fMRI. This has application in widening access to, and increasing ecological validity of, immersive neuroimaging across understanding, diagnosis, assessment and treatment of, a range of mental disorders such as phobia, anxiety and PTSD or addictions.

Stress assessment by means of heart rate derived from functional near-infrared spectroscopy

Many studies have been carried out in order to detect and quantify the level of mental stress by means of different physiological signals. From the physiological point of view, stress promptly affects brain and cardiac function; therefore, stress can be assessed by analyzing the brain- and heart-related signals more efficiently. Signals produced by functional near-infrared spectroscopy (fNIRS) of the brain together with the heart rate (HR) are employed to assess the stress induced by the Montreal Imaging Stress Task. Two different versions of the HR are used in this study. The first one is the commonly used HR derived from the electrocardiogram (ECG) and is considered as the reference HR (RHR). The other is the HR computed from the fNIRS signal (EHR) by means of an effective combinational algorithm. fNIRS and ECG signals were simultaneously recorded from 10 volunteers, and EHR and RHR are derived from them, respectively. Our results showed a high degree of agreement [r  >  0.9, BAR (Bland Altman ratio) <5  %  ] between the two HR. A principal component analysis/support vector machine-based algorithm for stress classification is developed and applied to the three measurements of fNIRS, EHR, and RHR and a classification accuracy of 78.8%, 94.6%, and 62.2% were obtained for the three measurements, respectively. From these observations, it can be concluded that the EHR carries more useful information with regards to the mental stress than the RHR and fNIRS signals. Therefore, EHR can be used alone or in combination with the fNIRS signal for a more accurate and real-time stress detection and classification.

Brain activity in response to the touch of a hand on the center of the back

The aim of this study was to validate the possibility of using functional Near-Infrared Spectroscopy (fNIRS) to measure changes in cerebral blood flow in response to a hand being placed on a participant's back, and to identify the areas of enhanced activity in the brain. Nineteen female adult volunteers participated in the study. An experienced school nurse touched the center of the participant's back between the shoulder blades with the palm of her hand. Cerebral blood volume dynamics were measured with a 52-channel fNIRS system. Significantly higher oxygenated hemoglobin (oxy-Hb) concentration levels were recorded by channels 11, 14, 21, 22, 24, 32, 35, 45, 46, and 49 during the touching period than during the resting period. These channels indicated enhanced activity in the supramarginal gyrus, the middle frontal gyrus, the superior temporal gyrus, and the inferior frontal gyrus. The ability to detect changes in cerebral blood flow using this method indicates the possibility of measuring changes in cerebral blood flow using fNIRS when a person is touched on the back. fNIRS has been shown to be useful for studying the effects of touch.

A Novel Method for Classifying Driver Mental Workload Under Naturalistic Conditions With Information From Near-Infrared Spectroscopy

Driver cognitive distraction is a critical factor in road safety, and its evaluation, especially under real conditions, presents challenges to researchers and engineers. In this study, we considered mental workload from a secondary task as a potential source of cognitive distraction and aimed to estimate the increased cognitive load on the driver with a four-channel near-infrared spectroscopy (NIRS) device by introducing a machine-learning method for hemodynamic data. To produce added cognitive workload in a driver beyond just driving, two levels of an auditory presentation n-back task were used. A total of 60 experimental data sets from the NIRS device during two driving tasks were obtained and analyzed by machine-learning algorithms. We used two techniques to prevent overfitting of the classification models: (1) k-fold cross-validation and principal-component analysis, and (2) retaining 25% of the data (testing data) for testing of the model after classification. Six types of classifier were trained and tested: decision tree, discriminant analysis, logistic regression, the support vector machine, the nearest neighbor classifier, and the ensemble classifier. Cognitive workload levels were well classified from the NIRS data in the cases of subject-dependent classification (the accuracy of classification increased from 81.30 to 95.40%, and the accuracy of prediction of the testing data was 82.18 to 96.08%), subject 26 independent classification (the accuracy of classification increased from 84.90 to 89.50%, and the accuracy of prediction of the testing data increased from 84.08 to 89.91%), and channel-independent classification (classification 82.90%, prediction 82.74%). NIRS data in conjunction with an artificial intelligence method can therefore be used to classify mental workload as a source of potential cognitive distraction in real time under naturalistic conditions; this information may be utilized in driver assistance systems to prevent road accidents.

Existence of Initial Dip for BCI: An Illusion or Reality

A tight coupling between the neuronal activity and the cerebral blood flow (CBF) is the motivation of many hemodynamic response (HR)-based neuroimaging modalities. The increase in neuronal activity causes the increase in CBF that is indirectly measured by HR modalities. Upon functional stimulation, the HR is mainly categorized in three durations: (i) initial dip, (ii) conventional HR (i.e., positive increase in HR caused by an increase in the CBF), and (iii) undershoot. The initial dip is a change in oxygenation prior to any subsequent increase in CBF and spatially more specific to the site of neuronal activity. Despite additional evidence from various HR modalities on the presence of initial dip in human and animal species (i.e., cat, rat, and monkey); the existence/occurrence of an initial dip in HR is still under debate. This article reviews the existence and elusive nature of the initial dip duration of HR in intrinsic signal optical imaging (ISOI), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The advent of initial dip and its elusiveness factors in ISOI and fMRI studies are briefly discussed. Furthermore, the detection of initial dip and its role in brain-computer interface using fNIRS is examined in detail. The best possible application for the initial dip utilization and its future implications using fNIRS are provided.

A Decoding Scheme for Incomplete Motor Imagery EEG With Deep Belief Network

High accuracy decoding of electroencephalogram (EEG) signal is still a major challenge that can hardly be solved in the design of an effective motor imagery-based brain-computer interface (BCI), especially when the signal contains various extreme artifacts and outliers arose from data loss. The conventional process to avoid such cases is to directly reject the entire severely contaminated EEG segments, which leads to a drawback that the BCI has no decoding results during that certain period. In this study, a novel decoding scheme based on the combination of Lomb-Scargle periodogram (LSP) and deep belief network (DBN) was proposed to recognize the incomplete motor imagery EEG. Particularly, instead of discarding the entire segment, two forms of data removal were adopted to eliminate the EEG portions with extreme artifacts and data loss. The LSP was utilized to steadily extract the power spectral density (PSD) features from the incomplete EEG constructed by the remaining portions. A DBN structure based on the restricted Boltzmann machine (RBM) was exploited and optimized to perform the classification task. Various comparative experiments were conducted and evaluated on simulated signal and real incomplete motor imagery EEG, including the comparison of three PSD extraction methods (fast Fourier transform, Welch and LSP) and two classifiers (DBN and support vector machine, SVM). The results demonstrate that the LSP can estimate relative robust PSD features and the proposed scheme can significantly improve the decoding performance for the incomplete motor imagery EEG. This scheme can provide an alternative decoding solution for the motor imagery EEG contaminated by extreme artifacts and data loss. It can be beneficial to promote the stability, smoothness and maintain consecutive outputs without interruption for a BCI system that is suitable for the online and long-term application.

Online classification of imagined speech using functional near-infrared spectroscopy signals

OBJECTIVE

Most brain-computer interfaces (BCIs) based on functional near-infrared spectroscopy (fNIRS) require that users perform mental tasks such as motor imagery, mental arithmetic, or music imagery to convey a message or to answer simple yes or no questions. These cognitive tasks usually have no direct association with the communicative intent, which makes them difficult for users to perform.

APPROACH

In this paper, a 3-class intuitive BCI is presented which enables users to directly answer yes or no questions by covertly rehearsing the word 'yes' or 'no' for 15 s. The BCI also admits an equivalent duration of unconstrained rest which constitutes the third discernable task. Twelve participants each completed one offline block and six online blocks over the course of two sessions. The mean value of the change in oxygenated hemoglobin concentration during a trial was calculated for each channel and used to train a regularized linear discriminant analysis (RLDA) classifier.

MAIN RESULTS

By the final online block, nine out of 12 participants were performing above chance (p  <  0.001 using the binomial cumulative distribution), with a 3-class accuracy of 83.8%  ±  9.4%. Even when considering all participants, the average online 3-class accuracy over the last three blocks was 64.1 %  ±  20.6%, with only three participants scoring below chance (p  <  0.001). For most participants, channels in the left temporal and temporoparietal cortex provided the most discriminative information.

SIGNIFICANCE

To our knowledge, this is the first report of an online 3-class imagined speech BCI. Our findings suggest that imagined speech can be used as a reliable activation task for selected users for development of more intuitive BCIs for communication.

On the Feasibility of Using an Ear-EEG to Develop an Endogenous Brain-Computer Interface

Brain-computer interface (BCI) studies based on electroencephalography (EEG) measured around the ears (ear-EEGs) have mostly used exogenous paradigms involving brain activity evoked by external stimuli. The objective of this study is to investigate the feasibility of ear-EEGs for development of an endogenous BCI system that uses self-modulated brain activity. We performed preliminary and main experiments where EEGs were measured on the scalp and behind the ears to check the reliability of ear-EEGs as compared to scalp-EEGs. In the preliminary and main experiments, subjects performed eyes-open and eyes-closed tasks, and they performed mental arithmetic (MA) and light cognitive (LC) tasks, respectively. For data analysis, the brain area was divided into four regions of interest (ROIs) (i.e., frontal, central, occipital, and ear area). The preliminary experiment showed that the degree of alpha activity increase of the ear area with eyes closed is comparable to those of other ROIs (occipital > ear > central > frontal). In the main experiment, similar event-related (de)synchronization (ERD/ERS) patterns were observed between the four ROIs during MA and LC, and all ROIs showed the mean classification accuracies above 70% required for effective binary communication (MA vs. LC) (occipital = ear = central = frontal). From the results, we demonstrated that ear-EEG can be used to develop an endogenous BCI system based on cognitive tasks without external stimuli, which allows the usability of ear-EEGs to be extended.

Wavelet-based method for removing global physiological noise in functional near-infrared spectroscopy

Functional near-infrared spectroscopy (fNIRS) is a fast-developing non-invasive functional brain imaging technology widely used in cognitive neuroscience, clinical research and neural engineering. However, it is a challenge to effectively remove the global physiological noise in the fNIRS signal. The global physiological noise in fNIRS arises from multiple physiological origins in both superficial tissues and the brain. It has complex temporal, spatial and frequency characteristics, casting significant influence on the results. In the present study, we developed a novel wavelet-based method for fNIRS global physiological noise removal. The method is data-driven and does not rely on any additional hardware or subjective noise component selection procedure. It consists of two steps. Firstly, we use wavelet transform coherence to automatically detect the time-frequency points contaminated by the global physiological noise. Secondly, we decompose the fNIRS signal by using the wavelet transform, and then suppress the wavelet energy of the contaminated time-frequency points. Finally, we transform the signal back to a time series. We validated the method by using simulation and real data at both task- and resting-state. The results showed that our method can effectively remove the global physiological noise from the fNIRS signal and improve the spatial specificity of the task activation and the resting-state functional connectivity pattern.

Prefrontal Cortex Oxygenation Evoked by Convergence Load Under Conflicting Stimulus-to-Accommodation and Stimulus-to-Vergence Eye-Movements Measured by NIRS

Purpose: To extend our knowledge of the functional linkages between visual fatigue and regional cerebral prefrontal cortex (PFC) oxygenation, we measured time related hemodynamic changes over the right dorsolateral prefrontal cortex (dlPFC) during convergence load under conflicting stimulus-to-accommodation and stimulus-to-vergence eye movements with and without concurrent mental load.

Methods: Twenty healthy participants with a median age of 28 years (range: 18–44 years) fixated upon a vertical bar presented separately to the left and right eyes, using polarized filters, during four counterbalanced 10-min periods: (i) no accommodation/vergence conflict (Control, Ctrl); (ii) added convergence load and accommodation/vergence conflict (Conv); (iii) added cognitive load only (Cog) and; (iv) a combination of added cognitive and convergence load and accommodation/vergence conflict (Cc). Viewing distance was 65 cm. Non-invasive measurements of hemodynamic activity over the dlPFC were quantified by functional near-infrared spectroscopy (fNIRS). During the two-convergence load conditions, the horizontal disparity of the two bars varied dynamically from no disparity to a disparity set 20% below the individual threshold for diplopia. Cognitive load was induced by the n-back-2 test which required the subject to memorize and recall the changing colors of the horizontal bars and decide when a given color was the same as that occurring two colors previously. fNIRS data were averaged over 10-s windows centered at 0, 2, 4, 6, 8, and 10 min of each task, subtracted from a 20-s baseline window immediately preceding the visual task, and then represented as changes in oxygenated hemoglobin (ΔHbO₂)_, deoxygenated hemoglobin (ΔHHb) and total hemoglobin (ΔtHb).

Results: Linear mixed model analyses showed that hemodynamic activity was systematically influenced by time (p < 0.001). The group-averaged time-related level of change across the viewing conditions did not differ when compared with one another (p > 0.05). Larger convergence eye-movement responses under conflicting stimulus-to-accommodation, and stimulus-to-vergence over time, increased ΔHbO₂ and ΔtHb only in condition Cc and after 8 min of task time (p < 0.10 for min^-6 and min^-8: p < 0.05 for min^-10).

Discussion: Collectively, our data suggest that HbO₂, HHb, and tHb, recorded over the dlPFC with fNIRS, can be used to assay the degree to which supervisory oculomotor control processes are activated during visually deficient near work.

Detection of Mental Task Related Activity in NIRS-BCI systems Using Dirichlet Energy over Graphs

Near Infrared Spectroscopy (NIRS)-based Brain Computer Interfaces (NIRS-BCI) rely mainly on the mean concentration changes and slope of the hemodynamic responses in separate recording channels to detect the mental-task related brain activity. Nevertheless, spatial patterns across the measurement channels are also present and should be taken into account for reliable evaluation of the aforementioned detection. In this work the Dirichlet Energy of NIRS signals over a graph is considered for the definition of a measure that would take into account the spatial NIRS features and would integrate the activity of multiple NIRS channels for robust mental task related activity detection. The application of the proposed measure on a real NIRS dataset demonstrates the efficiency of the proposed measure.

Classifying the mental representation of word meaning in children with Multivariate Pattern Analysis of fNIRS

This study presents the implementation of a within-subject neural decoder, based on Support Vector Machines, and its application for the classification of distributed patterns of hemodynamic activation, measured with Functional Near Infrared Spectroscopy (fNIRS) on children, in response to meaningful and meaningless auditory stimuli. Classification accuracy nominally exceeds chance level for the majority of the participants, but fails to reach statistical significance. Future work should investigate whether individual differences in classification accuracy may relate to other characteristics of the children, such as their cognitive, speech or reading abilities.

Comparison of source localization techniques in diffuse optical tomography for fNIRS application using a realistic head model

Functional near-infrared spectroscopy (fNIRS) is a non-invasive imaging technique that elicits growing interest for research and clinical applications. In the last decade, efforts have been made to develop a mathematical framework in order to image the effective sources of hemoglobin variations in brain tissues. Different approaches can be used to impose additional information or constraints when reconstructing the cerebral images of an ill-posed problem. The goal of this study is to compare the performance and limitations of several source localization techniques in the context of fNIRS tomography using individual anatomical magnetic resonance imaging (MRI) to model light propagation. The forward problem is solved using a Monte Carlo simulation of light propagation in the tissues. The inverse problem has been linearized using the Rytov approximation. Then, Tikhonov regularization applied to least squares, truncated singular value decomposition, back-projection, L1-norm regularization, minimum norm estimates, low resolution electromagnetic tomography and Bayesian model averaging techniques are compared using a receiver operating characteristic analysis, blurring and localization error measures. Using realistic simulations (n = 450) and data acquired from a human participant, this study depicts how these source localization techniques behave in a human head fNIRS tomography. When compared to other methods, Bayesian model averaging is proposed as a promising method in DOT and shows great potential to improve specificity, accuracy, as well as to reduce blurring and localization error even in presence of noise and deep sources. Classical reconstruction methods, such as regularized least squares, offer better sensitivity but higher blurring; while more novel L1-based method provides sparse solutions with small blurring and high specificity but lower sensitivity. The application of these methods is also demonstrated experimentally using visual fNIRS experiment with adult participant.

Functional near-infrared spectroscopy-based affective neurofeedback: feedback effect, illiteracy phenomena, and whole-connectivity profiles

Background: Affective neurofeedback constitutes a suitable approach to control abnormal neural activities associated with psychiatric disorders and might consequently relief symptom severity. However, different aspects of neurofeedback remain unclear, such as its neural basis, the performance variation, the feedback effect, among others. Aim: First, we aimed to propose a functional near-infrared spectroscopy (fNIRS)-based affective neurofeedback based on the self-regulation of frontal and occipital networks. Second, we evaluated three different feedback approaches on performance: real, fixed, and random feedback. Third, we investigated different demographic, psychological, and physiological predictors of performance. Approach: Thirty-three healthy participants performed a task whereby an amorphous figure changed its shape according to the elicited affect (positive or neutral). During the task, the participants randomly received three different feedback approaches: real feedback, with no change of the classifier output; fixed feedback, keeping the feedback figure unmodified; and random feedback, where the classifier output was multiplied by an arbitrary value, causing a feedback different than expected by the subject. Then, we applied a multivariate comparison of the whole-connectivity profiles according to the affective states and feedback approaches, as well as during a pretask resting-state block, to predict performance. Results: Participants were able to control this feedback system with 70.00 % ± 24.43 % ( p < 0.01 ) of performance during the real feedback trials. No significant differences were found when comparing the average performances of the feedback approaches. However, the whole functional connectivity profiles presented significant Mahalanobis distances ( p ≪ 0.001 ) when comparing both affective states and all feedback approaches. Finally, task performance was positively correlated to the pretask resting-state whole functional connectivity ( r = 0.512 , p = 0.009 ). Conclusions: Our results suggest that fNIRS might be a feasible tool to develop a neurofeedback system based on the self-regulation of affective networks. This finding enables future investigations using an fNIRS-based affective neurofeedback in psychiatric populations. Furthermore, functional connectivity profiles proved to be a good predictor of performance and suggested an increased effort to maintain task control in the presence of feedback distractors.

Classification of electroencephalogram signals using wavelet-CSP and projection extreme learning machine

Brain-computer interface (BCI) systems establish a direct communication channel from the brain to an output device. As the basis of BCIs, recognizing motor imagery activities poses a considerable challenge to signal processing due to the complex and non-stationary characteristics. This paper introduces an optimal and intelligent method for motor imagery BCIs. Because of the robustness to noise, wavelet packet decomposition and common spatial pattern (CSP) methods were implemented to reduce the dimensions of preprocessed signals. And a novel and efficient classifier projection extreme learning machine (PELM) was employed to recognize the labels of electroencephalogram signals. Experiments have been performed on the BCI Competition Dataset to demonstrate the superiority of wavelet-CSP in BCI and the outperformance of the PELM-based method. Results show that the average recognition rate of PELM approaches approximately 70%, while the optimal rate of other methods is 72%, whose training time and classification time are relatively longer as 11.00 ms and 11.66 ms, respectively, compared with 4.75 ms and 4.87 ms obtained by using the proposed BCI system.

Introducing chaos behavior to kernel relevance vector machine (RVM) for four-class EEG classification

This paper addresses a chaos kernel function for the relevance vector machine (RVM) in EEG signal classification, which is an important component of Brain-Computer Interface (BCI). The novel kernel function has evolved from a chaotic system, which is inspired by the fact that human brain signals depict some chaotic characteristics and behaviors. By introducing the chaotic dynamics to the kernel function, the RVM will be enabled for higher classification capacity. The proposed method is validated within the framework of one versus one common spatial pattern (OVO-CSP) classifier to classify motor imagination (MI) of four movements in a public accessible dataset. To illustrate the performance of the proposed kernel function, Gaussian and Polynomial kernel functions are considered for comparison. Experimental results show that the proposed kernel function achieved higher accuracy than Gaussian and Polynomial kernel functions, which shows that the chaotic behavior consideration is helpful in the EEG signal classification.

Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Neuronal Activation Detection Using Vector Phase Analysis with Dual Threshold Circles: A Functional Near-Infrared Spectroscopy Study

In this paper, a new vector phase diagram differentiating the initial decreasing phase (i.e. initial dip) and the delayed hemodynamic response (HR) phase of oxy-hemoglobin changes ( Δ HbO) of functional near-infrared spectroscopy (fNIRS) is developed. The vector phase diagram displays the trajectories of Δ HbO and deoxy-hemoglobin changes ( Δ HbR), as orthogonal components, in the Δ HbO- Δ HbR polar coordinates. To determine the occurrence of an initial dip, dual threshold circles (an inner circle from the resting state, an outer circle from the peak values of the initial dip and the main HR) are incorporated into the phase diagram for making decisions. The proposed scheme is then applied to a brain-computer interface scheme, and its performance is evaluated in classifying two finger tapping tasks (right-hand thumb and little finger) from the left motor cortex. Three gamma functions are used to model the initial dip, the main HR, and the undershoot in generating the designed HR function. In classifying two tapping tasks, the signal mean and signal minimum values during 0-2.5 s, as features of initial dip, are used. The linear discriminant analysis was utilized as a classifier. The experimental results show that the active brain locations of the two tasks were quite distinctive ( p < 0.05 ), and moreover, spatially specific if using the initial dip map at 4 s in comparison to the map of HRs at 14 s. Also, the average classification accuracy was improved from 59% to 74.9% when using the phase diagram of dual threshold circles.

Differential Path-Length Factor's Effect on the Characterization of Brain's Hemodynamic Response Function: A Functional Near-Infrared Study

Functional near-infrared spectroscopy (fNIRS) has evolved as a neuro-imaging modality over the course of the past two decades. The removal of superfluous information accompanying the optical signal, however, remains a challenge. A comprehensive analysis of each step is necessary to ensure the extraction of actual information from measured fNIRS waveforms. A slight change in shape could alter the features required for fNIRS-BCI applications. In the present study, the effect of the differential path-length factor (DPF) values on the characteristics of the hemodynamic response function (HRF) was investigated. Results were compiled for both simulated data sets and healthy human subjects over a range of DPF values from three to eight. Different sets of activation durations and stimuli were used to generate the simulated signals for further analysis. These signals were split into optical densities under a constrained environment utilizing known values of DPF. Later, different values of DPF were used to analyze the variations of actual HRF. The results, as summarized into four categories, suggest that the DPF can change the main and post-stimuli responses in addition to other interferences. Six healthy subjects participated in this study. Their observed optical brain time-series were fed into an iterative optimization problem in order to estimate the best possible fit of HRF and physiological noises present in the measured signals with free parameters. A series of solutions was derived for different values of DPF in order to analyze the variations of HRF. It was observed that DPF change is responsible for HRF creep from actual values as well as changes in HRF characteristics.

Use of Sine Shaped High-Frequency Rhythmic Visual Stimuli Patterns for SSVEP Response Analysis and Fatigue Rate Evaluation in Normal Subjects

Background: Recent EEG-SSVEP signal based BCI studies have used high frequency square pulse visual stimuli to reduce subjective fatigue. However, the effect of total harmonic distortion (THD) has not been considered. Compared to CRT and LCD monitors, LED screen displays high-frequency wave with better refresh rate. In this study, we present high frequency sine wave simple and rhythmic patterns with low THD rate by LED to analyze SSVEP responses and evaluate subjective fatigue in normal subjects. Materials and Methods: We used patterns of 3-sequence high-frequency sine waves (25, 30, and 35 Hz) to design our visual stimuli. Nine stimuli patterns, 3 simple (repetition of each of above 3 frequencies e.g., P25-25-25) and 6 rhythmic (all of the frequencies in 6 different sequences e.g., P25-30-35) were chosen. A hardware setup with low THD rate (<0.1%) was designed to present these patterns on LED. Twenty two normal subjects (aged 23-30 (25 ± 2.1) yrs) were enrolled. Visual analog scale (VAS) was used for subjective fatigue evaluation after presentation of each stimulus pattern. PSD, CCA, and LASSO methods were employed to analyze SSVEP responses. The data including SSVEP features and fatigue rate for different visual stimuli patterns were statistically evaluated. Results: All 9 visual stimuli patterns elicited SSVEP responses. Overall, obtained accuracy rates were 88.35% for PSD and > 90% for CCA and LASSO (for TWs > 1 s). High frequency rhythmic patterns group with low THD rate showed higher accuracy rate (99.24%) than simple patterns group (98.48%). Repeated measure ANOVA showed significant difference between rhythmic pattern features (P < 0.0005). Overall, there was no significant difference between the VAS of rhythmic [3.85 ± 2.13] compared to the simple patterns group [3.96 ± 2.21], (P = 0.63). Rhythmic group had lower within group VAS variation (min = P25-30-35 [2.90 ± 2.45], max = P35-25-30 [4.81 ± 2.65]) as well as least individual pattern VAS (P25-30-35). Discussion and Conclusion: Overall, rhythmic and simple pattern groups had higher and similar accuracy rates. Rhythmic stimuli patterns showed insignificantly lower fatigue rate than simple patterns. We conclude that both rhythmic and simple visual high frequency sine wave stimuli require further research for human subject SSVEP-BCI studies.

Mapping the Cortical Network Arising From Up-Regulated Amygdaloidal Activation Using -Louvain Algorithm

The amygdala plays an important role in emotion processing. Several studies have proved that its activation can be regulated by real-time functional magnetic resonance imaging (rtfMRI)-based neurofeedback training. However, although studies have found brain regions that are functionally closely connected to the amygdala in the cortex, it is not clear whether these brain regions and the amygdala are structurally closely connected, and if they show the same training effect as the amygdala in the process of emotional regulation. In this paper, we instructed subjects to up-regulate the activation of the left amygdala (LA) through rtfMRI-based neurofeedback training. In order to fuse multimodal imaging data, we introduced a network analysis method called the -Louvain clustering algorithm. This method was used to integrate multimodal data from the training experiment and construct an LA-cortical network. Correlation analysis and main-effect analysis were conducted to determine the signal covariance associated with the activation of the target area; ultimately, we identified the left temporal pole superior as the amygdaloidal-cortical network region. As a deep nucleus in the brain, the treatment and stimulation of the amygdala remains challenging. Our results provide new insights for the regulation of activation in a deep nucleus using more neurofeedback techniques.

Improvement of Information Transfer Rates Using a Hybrid EEG-NIRS Brain-Computer Interface with a Short Trial Length: Offline and Pseudo-Online Analyses

Electroencephalography (EEG) and near-infrared spectroscopy (NIRS) are non-invasive neuroimaging methods that record the electrical and metabolic activity of the brain, respectively. Hybrid EEG-NIRS brain-computer interfaces (hBCIs) that use complementary EEG and NIRS information to enhance BCI performance have recently emerged to overcome the limitations of existing unimodal BCIs, such as vulnerability to motion artifacts for EEG-BCI or low temporal resolution for NIRS-BCI. However, with respect to NIRS-BCI, in order to fully induce a task-related brain activation, a relatively long trial length (≥10 s) is selected owing to the inherent hemodynamic delay that lowers the information transfer rate (ITR; bits/min). To alleviate the ITR degradation, we propose a more practical hBCI operated by intuitive mental tasks, such as mental arithmetic (MA) and word chain (WC) tasks, performed within a short trial length (5 s). In addition, the suitability of the WC as a BCI task was assessed, which has so far rarely been used in the BCI field. In this experiment, EEG and NIRS data were simultaneously recorded while participants performed MA and WC tasks without preliminary training and remained relaxed (baseline; BL). Each task was performed for 5 s, which was a shorter time than previous hBCI studies. Subsequently, a classification was performed to discriminate MA-related or WC-related brain activations from BL-related activations. By using hBCI in the offline/pseudo-online analyses, average classification accuracies of 90.0 ± 7.1/85.5 ± 8.1% and 85.8 ± 8.6/79.5 ± 13.4% for MA vs. BL and WC vs. BL, respectively, were achieved. These were significantly higher than those of the unimodal EEG- or NIRS-BCI in most cases. Given the short trial length and improved classification accuracy, the average ITRs were improved by more than 96.6% for MA vs. BL and 87.1% for WC vs. BL, respectively, compared to those reported in previous studies. The suitability of implementing a more practical hBCI based on intuitive mental tasks without preliminary training and with a shorter trial length was validated when compared to previous studies.

Differential Entropy Preserves Variational Information of Near-Infrared Spectroscopy Time Series Associated With Working Memory

Neuroscience research shows a growing interest in the application of Near-Infrared Spectroscopy (NIRS) in analysis and decoding of the brain activity of human subjects. Given the correlation that is observed between the Blood Oxygen Dependent Level (BOLD) responses that are exhibited by the time series data of functional Magnetic Resonance Imaging (fMRI) and the hemoglobin oxy/deoxy-genation that is captured by NIRS, linear models play a central role in these applications. This, in turn, results in adaptation of the feature extraction strategies that are well-suited for discretization of data that exhibit a high degree of linearity, namely, slope and the mean as well as their combination, to summarize the informational contents of the NIRS time series. In this article, we demonstrate that these features are inefficient in capturing the variational information of NIRS data, limiting the reliability and the adequacy of the conclusion on their results. Alternatively, we propose the linear estimate of differential entropy of these time series as a natural representation of such information. We provide evidence for our claim through comparative analysis of the application of these features on NIRS data pertinent to several working memory tasks as well as naturalistic conversational stimuli.

Shining a Light on Awareness: A Review of Functional Near-Infrared Spectroscopy for Prolonged Disorders of Consciousness

Qualitative clinical assessments of the recovery of awareness after severe brain injury require an assessor to differentiate purposeful behavior from spontaneous behavior. As many such behaviors are minimal and inconsistent, behavioral assessments are susceptible to diagnostic errors. Advanced neuroimaging tools can bypass behavioral responsiveness and reveal evidence of covert awareness and cognition within the brains of some patients, thus providing a means for more accurate diagnoses, more accurate prognoses, and, in some instances, facilitated communication. The majority of reports to date have employed the neuroimaging methods of functional magnetic resonance imaging, positron emission tomography, and electroencephalography (EEG). However, each neuroimaging method has its own advantages and disadvantages (e.g., signal resolution, accessibility, etc.). Here, we describe a burgeoning technique of non-invasive optical neuroimaging—functional near-infrared spectroscopy (fNIRS)—and review its potential to address the clinical challenges of prolonged disorders of consciousness. We also outline the potential for simultaneous EEG to complement the fNIRS signal and suggest the future directions of research that are required in order to realize its clinical potential.

Wavelet Transform Analysis the Recognizing Brain Activities for Development the Palm-Size and Simplification Near-Infrared Spectroscopy Prototype System by Using Arduino

Functional near-infrared spectroscopy (fNIRS) and brain computer interface (BCI) have become indispensable tools for recording and monitoring brain activity, comprising a non-invasive and safe technique that allows researchers to monitor blood flow in the front part of the brain. Although some medical device manufacturers developed complex fNIRS systems, downsized fNIRS systems are important for other uses, such as in portable (palm-sized) and wearable healthcare devices. This paper proposes a downsized compact fNIRS prototype that detects hemodynamics in the frontal lobe. The aim is to develop a compact fNIRS system, which is reliable and easy to integrate into portable (palm-sized) BCI devices. Through practical experiments with human subjects, our proposed system showed an ability to detect and monitor the start and end time of human brain activities when participants were solving a calculation table.