fNIRS-based brain-computer interfaces: a review

Non-Invasive Functional Evaluation of the Human Spinal Cord by Assessing the Peri-Spinal Neurovascular Network With Near Infrared Spectroscopy

Current medical care lacks an effective functional evaluation for the spinal cord. Magnetic resonance imaging and computed tomography mainly provide structural information of the spinal cord, while spinal somatosensory evoked potentials are limited by a low signal to noise ratio. We developed a non-invasive approach based on near-infrared spectroscopy in dual-wavelength (760 and 850 nm for deoxy- or oxyhemoglobin respectively) to record the neurovascular response (NVR) of the peri-spinal vascular network at the 7th cervical and 10th thoracic vertebral levels of the spinal cord, triggered by unilateral median nerve electrical stimulation (square pulse, 5-10 mA, 5 ms, 1 pulse every 4 minutes) at the wrist. Amplitude, rise-time, and duration of NVR were characterized in 20 healthy participants. A single, painless stimulus was able to elicit a high signal-to-noise ratio and multi-segmental NVR (mainly from Oxyhemoglobin) with a fast rise time of 6.18 [4.4-10.4] seconds (median [Percentile 25-75]) followed by a slow decay phase for about 30 seconds toward the baseline. Cervical NVR was earlier and larger than thoracic and no left/right asymmetry was detected. Stimulus intensity/NVR amplitude fitted to a 2nd order function. The characterization and feasibility of the peri-spinal NVR strongly support the potential clinical applications for a functional assessment of spinal cord lesions.

Sleep deprivation impairs cognitive performance, alters task-associated cerebral blood flow and decreases cortical neurovascular coupling-related hemodynamic responses

Sleep deprivation (SD) is a common condition and an important health concern. In addition to metabolic and cardiovascular risks, SD associates with decreases in cognitive performance. Neurovascular coupling (NVC, "functional hyperemia") is a critical homeostatic mechanism, which maintains adequate blood supply to the brain during periods of intensive neuronal activity. To determine whether SD alters NVC responses and cognitive performance, cognitive and hemodynamic NVC assessments were conducted prior to and 24 h post-SD in healthy young male individuals (n = 10, 27 ± 3 years old). Cognition was evaluated with a battery of tests from the Cambridge Neuropsychological Test Automated Battery (CANTAB). Hemodynamic components of NVC were measured by transcranial Doppler sonography (TCD) during cognitive stimulation, dynamic retinal vessel analysis (DVA) during flicker light stimulation, and functional near infrared spectroscopy (fNIRS) during finger tapping motor task. Cognitive assessments revealed impairments in reaction time and sustained attention after 24 h of SD. Functional NIRS analysis revealed that SD significantly altered hemodynamic responses in the prefrontal cortex and somatosensory cortex during a motor task. NVC-related vascular responses measured by DVA and TCD did not change significantly. Interestingly, TCD detected decreased task-associated cerebral blood flow (CBF) in the right middle cerebral artery in sleep deprived participants. Our results demonstrate that 24 h of SD lead to impairments in cognitive performance together with altered CBF and hemodynamic components of cortical NVC responses.

Brain–Computer Interfacing Using Functional Near-Infrared Spectroscopy (fNIRS)

Functional Near-Infrared Spectroscopy (fNIRS) is a wearable optical spectroscopy system originally developed for continuous and non-invasive monitoring of brain function by measuring blood oxygen concentration. Recent advancements in brain–computer interfacing allow us to control the neuron function of the brain by combining it with fNIRS to regulate cognitive function. In this review manuscript, we provide information regarding current advancement in fNIRS and how it provides advantages in developing brain–computer interfacing to enable neuron function. We also briefly discuss about how we can use this technology for further applications.

Prefrontal Cortex Activation During Verbal Fluency Task and Tower of London Task in Schizophrenia and Major Depressive Disorder

Background: Cognitive dysfunction is a common clinical feature of mental disorders. A number of functional near-infrared spectroscopy (fNIRS) studies have shown reduced prefrontal activation during the verbal fluency task (VFT) in schizophrenia (SZ) and major depressive disorder (MDD). However, no studies have examined and compared the brain activation patterns during the Tower of London (TOL), which is another classic, high-sensitivity executive function testing tool, in these two serious mental disorders. This study aimed to assess the characteristics of brain activation during the two different cognitive tasks in SZ and MDD patients. Methods: This study recruited 30 patients with SZ, 30 patients with MDD, and 30 demographically matched healthy controls (HCs). The hemodynamic changes of the prefrontal cortex (PFC) were measured using 32-channel fNIRS during performance of the TOL task and VFT task. Results: SZ patients showed poorer VFT performance than MDD patients and HCs, and the two patient groups showed poorer TOL performance than HCs. Compared to HCs, both of the patient groups exhibited a significant decreased activation in the extensive PFC. Particularly in certain channels in the dorsolateral PFC (DLPFC), SZ patients exhibited significantly decreased hemodynamic changes than the MDD patients. Conclusions: Patients with SZ and MDD have different levels of impairment in different cognitive domains and different patterns of brain activation during the two cognitive tasks. Further research is needed to determine the use of fNIRS for clinical evaluation and diagnosis.

Most favorable stimulation duration in the sensorimotor cortex for fNIRS-based BCI

One of the primary objectives of the brain-computer interface (BCI) is to obtain a command with higher classification accuracy within the shortest possible time duration. Therefore, this study evaluates several stimulation durations to propose a duration that can yield the highest classification accuracy. Furthermore, this study aims to address the inherent delay in the hemodynamic responses (HRs) for the command generation time. To this end, HRs in the sensorimotor cortex were evaluated for the functional near-infrared spectroscopy (fNIRS)-based BCI. To evoke brain activity, right-hand-index finger poking and tapping tasks were used. In this study, six different stimulation durations (i.e., 1, 3, 5, 7, 10, and 15 s) were tested on 10 healthy male subjects. Upon stimulation, different temporal features and multiple time windows were utilized to extract temporal features. The extracted features were then classified using linear discriminant analysis. The classification results using the main HR showed that a 5 s stimulation duration could yield the highest classification accuracy, i.e., 74%, with a combination of the mean and maximum value features. However, the results were not significantly different from the classification accuracy obtained using the 15 s stimulation. To further validate the results, a classification using the initial dip was performed. The results obtained endorsed the finding with an average classification accuracy of 73.5% using the features of minimum peak and skewness in the 5 s window. The results based on classification using the initial dip for 5 s were significantly different from all other tested stimulation durations (p < 0.05) for all feature combinations. Moreover, from the visual inspection of the HRs, it is observed that the initial dip occurred as soon as the task started, but the main HR had a delay of more than 2 s. Another interesting finding is that impulsive stimulation in the sensorimotor cortex can result in the generation of a clearer initial dip phenomenon. The results reveal that the command for the fNIRS-based BCI can be generated using the 5 s stimulation duration. In conclusion, the use of the initial dip can reduce the time taken for the generation of commands and can be used to achieve a higher classification accuracy for the fNIRS-BCI within a 5 s task duration rather than relying on longer durations.

Comparison of functional activation responses from the auditory cortex derived using multi-distance frequency domain and continuous wave near-infrared spectroscopy

Significance: Quantitative measurements of cerebral hemodynamic changes due to functional activation are widely accomplished with commercial continuous wave (CW-NIRS) instruments despite the availability of the more rigorous multi-distance frequency domain (FD-NIRS) approach. A direct comparison of the two approaches to functional near-infrared spectroscopy can help in the interpretation of optical data and guide implementations of diffuse optical instruments for measuring functional activation. Aim: We explore the differences between CW-NIRS and multi-distance FD-NIRS by comparing measurements of functional activation in the human auditory cortex. Approach: Functional activation of the human auditory cortex was measured using a commercial frequency domain near-infrared spectroscopy instrument for 70 dB sound pressure level broadband noise and pure tone (1000 Hz) stimuli. Changes in tissue oxygenation were calculated using the modified Beer-Lambert law (CW-NIRS approach) and the photon diffusion equation (FD-NIRS approach). Results: Changes in oxygenated hemoglobin measured with the multi-distance FD-NIRS approach were about twice as large as those measured with the CW-NIRS approach. A finite-element simulation of the functional activation problem was performed to demonstrate that tissue oxygenation changes measured with the CW-NIRS approach is more accurate than that with multi-distance FD-NIRS. Conclusions: Multi-distance FD-NIRS approaches tend to overestimate functional activation effects, in part due to partial volume effects.

A Novel Spectrum Contrast Mapping Method for Functional Magnetic Resonance Imaging Data Analysis

Many studies reported that spontaneous fluctuation of the blood oxygen level-dependent signal exists in multiple frequency components and changes over time. By assuming a reliable energy contrast between low- and high-frequency bands for each voxel, we developed a novel spectrum contrast mapping (SCM) method to decode brain activity at the voxel-wise level and further validated it in designed experiments. SCM consists of the following steps: first, the time course of each given voxel is subjected to fast Fourier transformation; the corresponding spectrum is divided into low- and high-frequency bands by given reference frequency points; then, the spectral energy ratio of the low- to high-frequency bands is calculated for each given voxel. Finally, the activity decoding map is formed by the aforementioned energy contrast values of each voxel. Our experimental results demonstrate that the SCM (1) was able to characterize the energy contrast of task-related brain regions; (2) could decode brain activity at rest, as validated by the eyes-closed and eyes-open resting-state experiments; (3) was verified with test-retest validation, indicating excellent reliability with most coefficients > 0.9 across the test sessions; and (4) could locate the aberrant energy contrast regions which might reveal the brain pathology of brain diseases, such as Parkinson’s disease. In summary, we demonstrated that the reliable energy contrast feature was a useful biomarker in characterizing brain states, and the corresponding SCM showed excellent brain activity-decoding performance at the individual and group levels, implying its potentially broad application in neuroscience, neuroimaging, and brain diseases.

Current Review of Optical Neural Interfaces for Clinical Applications

Neural interfaces, which enable the recording and stimulation of living neurons, have emerged as valuable tools in understanding the brain in health and disease, as well as serving as neural prostheses. While neural interfaces are typically based on electrical transduction, alternative energy modalities have been explored to create safe and effective approaches. Among these approaches, optical methods of linking neurons to the outside world have gained attention because light offers high spatial selectivity and decreased invasiveness. Here, we review the current state-of-art of optical neural interfaces and their clinical applications. Optical neural interfaces can be categorized into optical control and optical readout, each of which can be divided into intrinsic and extrinsic approaches. We discuss the advantages and disadvantages of each of these methods and offer a comparison of relative performance. Future directions, including their clinical opportunities, are discussed with regard to the optical properties of biological tissue.

Conditional-GAN Based Data Augmentation for Deep Learning Task Classifier Improvement Using fNIRS Data

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique used for mapping the functioning human cortex. fNIRS can be widely used in population studies due to the technology’s economic, non-invasive, and portable nature. fNIRS can be used for task classification, a crucial part of functioning with Brain-Computer Interfaces (BCIs). fNIRS data are multidimensional and complex, making them ideal for deep learning algorithms for classification. Deep Learning classifiers typically need a large amount of data to be appropriately trained without over-fitting. Generative networks can be used in such cases where a substantial amount of data is required. Still, the collection is complex due to various constraints. Conditional Generative Adversarial Networks (CGAN) can generate artificial samples of a specific category to improve the accuracy of the deep learning classifier when the sample size is insufficient. The proposed system uses a CGAN with a CNN classifier to enhance the accuracy through data augmentation. The system can determine whether the subject’s task is a Left Finger Tap, Right Finger Tap, or Foot Tap based on the fNIRS data patterns. The authors obtained a task classification accuracy of 96.67% for the CGAN-CNN combination.

The Potential Role of fNIRS in Evaluating Levels of Consciousness

Over the last few decades, neuroimaging techniques have transformed our understanding of the brain and the effect of neurological conditions on brain function. More recently, light-based modalities such as functional near-infrared spectroscopy have gained popularity as tools to study brain function at the bedside. A recent application is to assess residual awareness in patients with disorders of consciousness, as some patients retain awareness albeit lacking all behavioural response to commands. Functional near-infrared spectroscopy can play a vital role in identifying these patients by assessing command-driven brain activity. The goal of this review is to summarise the studies reported on this topic, to discuss the technical and ethical challenges of working with patients with disorders of consciousness, and to outline promising future directions in this field.

Optimization of Task Allocation for Collaborative Brain-Computer Interface Based on Motor Imagery

OBJECTIVE

Collaborative brain-computer interfaces (cBCIs) can make the BCI output more credible by jointly decoding concurrent brain signals from multiple collaborators. Current cBCI systems usually require all collaborators to execute the same mental tasks (common-work strategy). However, it is still unclear whether the system performance will be improved by assigning different tasks to collaborators (division-of-work strategy) while keeping the total tasks unchanged. Therefore, we studied a task allocation scheme of division-of-work and compared the corresponding classification accuracies with common-work strategy's.

APPROACH

This study developed an electroencephalograph (EEG)-based cBCI which had six instructions related to six different motor imagery tasks (MI-cBCI), respectively. For the common-work strategy, all five subjects as a group had the same whole instruction set and they were required to conduct the same instruction at a time. For the division-of-work strategy, every subject's instruction set was a subset of the whole one and different from each other. However, their union set was equal to the whole set. Based on the number of instructions in a subset, we divided the division-of-work strategy into four types, called "2 Tasks" … "5 Tasks." To verify the effectiveness of these strategies, we employed EEG data collected from 19 subjects who independently performed six types of MI tasks to conduct the pseudo-online classification of MI-cBCI.

MAIN RESULTS

Taking the number of tasks performed by one collaborator as the horizontal axis (two to six), the classification accuracy curve of MI-cBCI was mountain-like. The curve reached its peak at "4 Tasks," which means each subset contained four instructions. It outperformed the common-work strategy ("6 Tasks") in classification accuracy (72.29 ± 4.43 vs. 58.53 ± 4.36%).

SIGNIFICANCE

The results demonstrate that our proposed task allocation strategy effectively enhanced the cBCI classification performance and reduced the individual workload.

Functional Brain Imaging Reliably Predicts Bimanual Motor Skill Performance in a Standardized Surgical Task

Currently, there is a dearth of objective metrics for assessing bi-manual motor skills, which are critical for high-stakes professions such as surgery. Recently, functional near-infrared spectroscopy (fNIRS) has been shown to be effective at classifying motor task types, which can be potentially used for assessing motor performance level. In this work, we use fNIRS data for predicting the performance scores in a standardized bi-manual motor task used in surgical certification and propose a deep-learning framework 'Brain-NET' to extract features from the fNIRS data. Our results demonstrate that the Brain-NET is able to predict bi-manual surgical motor skills based on neuroimaging data accurately ( R2=0.73). Furthermore, the classification ability of the Brain-NET model is demonstrated based on receiver operating characteristic (ROC) curves and area under the curve (AUC) values of 0.91. Hence, these results establish that fNIRS associated with deep learning analysis is a promising method for a bedside, quick and cost-effective assessment of bi-manual skill levels.

Recognition of Flexion and Extension Imagery Involving the Right and Left Arms Based on Deep Belief Network and Functional Near-Infrared Spectroscopy

Brain-computer interaction based on motor imagery (MI) is an important brain-computer interface (BCI). Most methods for MI classification are based on electroencephalogram (EEG), and few studies have investigated signal processing based on MI-Functional Near-Infrared Spectroscopy (fNIRS). In addition, there is a need to improve the classification accuracy for MI fNIRS methods. In this study, a deep belief network (DBN) based on a restricted Boltzmann machine (RBM) was used to classify fNIRS signals of flexion and extension imagery involving the left and right arms. fNIRS signals from 16 channels covering the motor cortex area were recorded for each of 10 subjects executing or imagining flexion and extension involving the left and right arms. Oxygenated hemoglobin (HbO) concentration was used as a feature to train two RBMs that were subsequently stacked with an additional softmax regression output layer to construct DBN. We also explored the DBN model classification accuracy for the test dataset from one subject using training dataset from other subjects. The average DBN classification accuracy for flexion and extension movement and imagery involving the left and right arms was 84.35 ± 3.86% and 78.19 ± 3.73%, respectively. For a given DBN model, better classification results are obtained for test datasets for a given subject when the model is trained using dataset from the same subject than when the model is trained using datasets from other subjects. The results show that the DBN algorithm can effectively identify flexion and extension imagery involving the right and left arms using fNIRS. This study is expected to serve as a reference for constructing online MI-BCI systems based on DBN and fNIRS.

Decoding different working memory states during an operation span task from prefrontal fNIRS signals

We propose an effective and practical decoding method of different mental states for potential applications for the design of brain-computer interfaces, prediction of cognitive behaviour, and investigation of cognitive mechanism. Functional near infrared spectroscopy (fNIRS) signals that interrogated the prefrontal and parietal cortices and were evaluated by generalized linear model were recorded when nineteen healthy adults performed the operation span (OSPAN) task. The oxygenated hemoglobin changes during OSPAN, response, and rest periods were classified with a support vector machine (SVM). The relevance vector regression algorithm was utilized for prediction of cognitive performance based on multidomain features of fNIRS signals from the OSPAN task. We acquired decent classification accuracies for OSPAN vs. response (above 91.2%) and for OSPAN vs. rest (above 94.7%). Eight of the ten cognitive testing scores could be predicted from the combination of OSPAN and response features, which indicated the brain hemodynamic responses contain meaningful information suitable for predicting cognitive performance.

Prefrontal Cortex Activation During Motor Sequence Learning Under Interleaved and Repetitive Practice: A Two-Channel Near-Infrared Spectroscopy Study

Training under high interference conditions through interleaved practice (IP) results in performance suppression during training but enhances long-term performance relative to repetitive practice (RP) involving low interference. Previous neuroimaging work addressing this contextual interference effect of motor learning has relied heavily on the blood-oxygen-level-dependent (BOLD) response using functional magnetic resonance imaging (fMRI) methodology resulting in mixed reports of prefrontal cortex (PFC) recruitment under IP and RP conditions. We sought to clarify these equivocal findings by imaging bilateral PFC recruitment using functional near-infrared spectroscopy (fNIRS) while discrete key pressing sequences were trained under IP and RP schedules and subsequently tested following a 24-h delay. An advantage of fNIRS over the fMRI BOLD response is that the former measures oxygenated and deoxygenated hemoglobin changes independently allowing for assessment of cortical hemodynamics even when there is neurovascular decoupling. Despite slower sequence performance durations under IP, bilateral PFC oxygenated and deoxygenated hemoglobin values did not differ between practice conditions. During test, however, slower performance from those previously trained under RP coincided with hemispheric asymmetry in PFC recruitment. Specifically, following RP, test deoxygenated hemoglobin values were significantly lower in the right PFC. The present findings contrast with previous behavioral demonstrations of increased cognitive demand under IP to illustrate a more complex involvement of the PFC in the contextual interference effect. IP and RP incur similar levels of bilateral PFC recruitment, but the processes underlying the recruitment are dissimilar. PFC recruitment during IP supports action reconstruction and memory elaboration while RP relies on PFC recruitment to maintain task variation information in working memory from trial to trial. While PFC recruitment under RP serves to enhance immediate performance, it does not support long-term performance.

Data Processing in Functional Near-Infrared Spectroscopy (fNIRS) Motor Control Research

FNIRS pre-processing and processing methodologies are very important-how a researcher chooses to process their data can change the outcome of an experiment. The purpose of this review is to provide a guide on fNIRS pre-processing and processing techniques pertinent to the field of human motor control research. One hundred and twenty-three articles were selected from the motor control field and were examined on the basis of their fNIRS pre-processing and processing methodologies. Information was gathered about the most frequently used techniques in the field, which included frequency cutoff filters, wavelet filters, smoothing filters, and the general linear model (GLM). We discuss the methodologies of and considerations for these frequently used techniques, as well as those for some alternative techniques. Additionally, general considerations for processing are discussed.

Functional interferometric diffusing wave spectroscopy of the human brain

Cerebral blood flow (CBF) is essential for brain function, and CBF-related signals can inform us about brain activity. Yet currently, high-end medical instrumentation is needed to perform a CBF measurement in adult humans. Here, we describe functional interferometric diffusing wave spectroscopy (fiDWS), which introduces and collects near-infrared light via the scalp, using inexpensive detector arrays to rapidly monitor coherent light fluctuations that encode brain blood flow index (BFI), a surrogate for CBF. Compared to other functional optical approaches, fiDWS measures BFI faster and deeper while also providing continuous wave absorption signals. Achieving clear pulsatile BFI waveforms at source-collector separations of 3.5 cm, we confirm that optical BFI, not absorption, shows a graded hypercapnic response consistent with human cerebrovascular physiology, and that BFI has a better contrast-to-noise ratio than absorption during brain activation. By providing high-throughput measurements of optical BFI at low cost, fiDWS will expand access to CBF.

Exploring the SenseMaking Process through Interactions and fNIRS in Immersive Visualization

Theories of cognition inform our decisions when designing human-computer interfaces, and immersive systems enable us to examine these theories. This work explores the sensemaking process in an immersive environment through studying both internal and external user behaviors with a classical visualization problem: a visual comparison and clustering task. We developed an immersive system to perform a user study, collecting user behavior data from different channels: AR HMD for capturing external user interactions, functional near-infrared spectroscopy (fNIRS) for capturing internal neural sequences, and video for references. To examine sensemaking, we assessed how the layout of the interface (planar 2D vs. cylindrical 3D layout) and the challenge level of the task (low vs. high cognitive load) influenced the users' interactions, how these interactions changed over time, and how they influenced task performance. We also developed a visualization system to explore joint patterns among all the data channels. We found that increased interactions and cerebral hemodynamic responses were associated with more accurate performance, especially on cognitively demanding trials. The layout types did not reliably influence interactions or task performance. We discuss how these findings inform the design and evaluation of immersive systems, predict user performance and interaction, and offer theoretical insights about sensemaking from the perspective of embodied and distributed cognition.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Effect of force accuracy on hemodynamic response: an fNIRS study using fine visuomotor task

Objective. Despite converging neuroimaging studies investigating how neural activity is modulated by various motor related factors, such as movement velocity and force magnitude, little has been devoted to identifying the effect of force accuracy. This study thus aimed to investigate the effect of task difficulty on cortical neural responses when participants performed a visuomotor task with varying demands on force accuracy.Approach. Fourteen healthy adults performed a set of force generation operations with six levels of force accuracy. The participants held a pen-shaped tool and moved the tool along a planar ring path, meanwhile producing a constant force against the plane under visual guidance. The required force accuracy was modulated by allowable tolerance of the force during the task execution. We employed functional near-infrared spectroscopy to record signals from bilateral prefrontal, sensorimotor and occipital areas, used the hemoglobin concentration as indicators of cortical activation, then calculated the effective connectivity across these regions by Granger causality.Main results.We observed overall stronger activation (oxy-hemoglobin concentration,p= 0.015) and connectivity (p< 0.05) associated with the initial increase in force accuracy, and the diminished trend in activation and connectivity when participants were exposed to excessive demands on accurate force generation. These findings suggested that the increasing task difficulty would be only beneficial for the mental investment up to a certain point, and above that point neural responses would show patterns of lower activation and connections, revealing mental overload at excessive task demands.Significance.Our results provide the first evidence for the inverted U-shaped effect of force accuracy on hemodynamic responses during fine visuomotor tasks. The insights obtained through this study also highlight the essential role of inter-region connectivity alterations for coping with task difficulty, enhance our understanding of the underlying motor neural processes, and provide the groundwork for developing adaptive neurorehabilitation strategies.

CNN-based classification of fNIRS signals in motor imagery BCI system

Objective. Development of a brain-computer interface (BCI) requires classification of brain neural activities to different states. Functional near-infrared spectroscopy (fNIRS) can measure the brain activities and has great potential for BCI. In recent years, a large number of classification algorithms have been proposed, in which deep learning methods, especially convolutional neural network (CNN) methods are successful. fNIRS signal has typical time series properties, we combined fNIRS data and kinds of CNN-based time series classification (TSC) methods to classify BCI task.Approach. In this study, participants were recruited for a left and right hand motor imagery experiment and the cerebral neural activities were recorded by fNIRS equipment (FOIRE-3000). TSC methods are used to distinguish the brain activities when imagining the left or right hand. We have tested the overall person, single person and overall person with single-channel classification results, and these methods achieved excellent classification results. We also compared the CNN-based TSC methods with traditional classification methods such as support vector machine.Main results. Experiments showed that the CNN-based methods have significant advantages in classification accuracy: the CNN-based methods have achieved remarkable results in the classification of left-handed and right-handed imagination tasks, reaching 98.6% accuracy on overall person, 100% accuracy on single person, and in the single-channel classification an accuracy of 80.1% has been achieved with the best-performing channel.Significance. These results suggest that using the CNN-based TSC methods can significantly improve the BCI performance and also lay the foundation for the miniaturization and portability of training rehabilitation equipment.

Guiding functional near-infrared spectroscopy optode-layout design using individual (f)MRI data: effects on signal strength

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain-computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.

Evidence of Neurovascular Un-Coupling in Mild Alzheimer's Disease through Multimodal EEG-fNIRS and Multivariate Analysis of Resting-State Data

Alzheimer’s disease (AD) is associated with modifications in cerebral blood perfusion and autoregulation. Hence, neurovascular coupling (NC) alteration could become a biomarker of the disease. NC might be assessed in clinical settings through multimodal electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Multimodal EEG-fNIRS was recorded at rest in an ambulatory setting to assess NC and to evaluate the sensitivity and specificity of the methodology to AD. Global NC was evaluated with a general linear model (GLM) framework by regressing whole-head EEG power envelopes in three frequency bands (theta, alpha and beta) with average fNIRS oxy- and deoxy-hemoglobin concentration changes in the frontal and prefrontal cortices. NC was lower in AD compared to healthy controls (HC) with significant differences in the linkage of theta and alpha bands with oxy- and deoxy-hemoglobin, respectively (p = 0.028 and p = 0.020). Importantly, standalone EEG and fNIRS metrics did not highlight differences between AD and HC. Furthermore, a multivariate data-driven analysis of NC between the three frequency bands and the two hemoglobin species delivered a cross-validated classification performance of AD and HC with an Area Under the Curve, AUC = 0.905 (p = 2.17 × 10⁻⁵). The findings demonstrate that EEG-fNIRS may indeed represent a powerful ecological tool for clinical evaluation of NC and early identification of AD.

Systemic Review on Transcranial Electrical Stimulation Parameters and EEG/fNIRS Features for Brain Diseases

Background

Brain disorders are gradually becoming the leading cause of death worldwide. However, the lack of knowledge of brain disease’s underlying mechanisms and ineffective neuropharmacological therapy have led to further exploration of optimal treatments and brain monitoring techniques.

Objective

This study aims to review the current state of brain disorders, which utilize transcranial electrical stimulation (tES) and daily usable noninvasive neuroimaging techniques. Furthermore, the second goal of this study is to highlight available gaps and provide a comprehensive guideline for further investigation.

Method

A systematic search was conducted of the PubMed and Web of Science databases from January 2000 to October 2020 using relevant keywords. Electroencephalography (EEG) and functional near-infrared spectroscopy were selected as noninvasive neuroimaging modalities. Nine brain disorders were investigated in this study, including Alzheimer’s disease, depression, autism spectrum disorder, attention-deficit hyperactivity disorder, epilepsy, Parkinson’s disease, stroke, schizophrenia, and traumatic brain injury.

Results

Sixty-seven studies (1,385 participants) were included for quantitative analysis. Most of the articles (82.6%) employed transcranial direct current stimulation as an intervention method with modulation parameters of 1 mA intensity (47.2%) for 16–20 min (69.0%) duration of stimulation in a single session (36.8%). The frontal cortex (46.4%) and the cerebral cortex (47.8%) were used as a neuroimaging modality, with the power spectrum (45.7%) commonly extracted as a quantitative EEG feature.

Conclusion

An appropriate stimulation protocol applying tES as a therapy could be an effective treatment for cognitive and neurological brain disorders. However, the optimal tES criteria have not been defined; they vary across persons and disease types. Therefore, future work needs to investigate a closed-loop tES with monitoring by neuroimaging techniques to achieve personalized therapy for brain disorders.

Defining Surgical Terminology and Risk for Brain Computer Interface Technologies

With the emergence of numerous brain computer interfaces (BCI), their form factors, and clinical applications the terminology to describe their clinical deployment and the associated risk has been vague. The terms “minimally invasive” or “non-invasive” have been commonly used, but the risk can vary widely based on the form factor and anatomic location. Thus, taken together, there needs to be a terminology that best accommodates the surgical footprint of a BCI and their attendant risks. This work presents a semantic framework that describes the BCI from a procedural standpoint and its attendant clinical risk profile. We propose extending the common invasive/non-invasive distinction for BCI systems to accommodate three categories in which the BCI anatomically interfaces with the patient and whether or not a surgical procedure is required for deployment: (1) Non-invasive—BCI components do not penetrate the body, (2) Embedded—components are penetrative, but not deeper than the inner table of the skull, and (3) Intracranial –components are located within the inner table of the skull and possibly within the brain volume. Each class has a separate risk profile that should be considered when being applied to a given clinical population. Optimally, balancing this risk profile with clinical need provides the most ethical deployment of these emerging classes of devices. As BCIs gain larger adoption, and terminology becomes standardized, having an improved, more precise language will better serve clinicians, patients, and consumers in discussing these technologies, particularly within the context of surgical procedures.

Investigating the Viability of Motor Imagery as a Physical Rehabilitation Treatment for Patients With Stroke-Induced Motor Cortical Damage

Although around 83% of individuals survive a stroke, they usually experience a significant loss in their motor execution (ME) capabilities due to their acquired cortical infarction. The loss of significant ME capabilities due to stroke damage was previously thought to be irreversible. Active movement therapies show considerable promise but depend on motor performance, excluding many otherwise eligible patients. Motor imagery (MI), a process that involves the use of mirror neurons to imagine motor activity, has emerged as a possible avenue to re-acquire some physical abilities lost to stroke damage. This paper examines previous studies to compare the strength of brain activation and connectivity in individuals who have brain lesions and those who do not as they all attempt ME and MI tasks. This paper reviews case studies investigating the direct effect of motor imagery in conjunction with physical therapy and the limitations of motor imagery based on the location of cortical damage and other variables, such as age. The findings analyzed in this review indicate that MI would serve as a beneficial addition to physical therapy and a viable option to stimulate motor evoked potentials (MEPs) in individuals not capable of pursuing physical therapy due to severe motor impairment. Regardless of the presence of brain lesions, motor imagery has consistently had a positive impact on motor rehabilitation either in boosting treatment or stimulating neuromuscular pathways. Therefore, we have concluded that MI is a viable supplemental treatment plan for motor recovery in most patients with motor cortical atrophy.

Motor Training Using Mental Workload (MWL) With an Assistive Soft Exoskeleton System: A Functional Near-Infrared Spectroscopy (fNIRS) Study for Brain-Machine Interface (BMI)

Mental workload is a neuroergonomic human factor, which is widely used in planning a system's safety and areas like brain-machine interface (BMI), neurofeedback, and assistive technologies. Robotic prosthetics methodologies are employed for assisting hemiplegic patients in performing routine activities. Assistive technologies' design and operation are required to have an easy interface with the brain with fewer protocols, in an attempt to optimize mobility and autonomy. The possible answer to these design questions may lie in neuroergonomics coupled with BMI systems. In this study, two human factors are addressed: designing a lightweight wearable robotic exoskeleton hand that is used to assist the potential stroke patients with an integrated portable brain interface using mental workload (MWL) signals acquired with portable functional near-infrared spectroscopy (fNIRS) system. The system may generate command signals for operating a wearable robotic exoskeleton hand using two-state MWL signals. The fNIRS system is used to record optical signals in the form of change in concentration of oxy and deoxygenated hemoglobin (HbO and HbR) from the pre-frontal cortex (PFC) region of the brain. Fifteen participants participated in this study and were given hand-grasping tasks. Two-state MWL signals acquired from the PFC region of the participant's brain are segregated using machine learning classifier-support vector machines (SVM) to utilize in operating a robotic exoskeleton hand. The maximum classification accuracy is 91.31%, using a combination of mean-slope features with an average information transfer rate (ITR) of 1.43. These results show the feasibility of a two-state MWL (fNIRS-based) robotic exoskeleton hand (BMI system) for hemiplegic patients assisting in the physical grasping tasks.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Based on Convolutional Neural Networks

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain–computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

High density optical neuroimaging predicts surgeons's subjective experience and skill levels

Measuring cognitive load is important for surgical education and patient safety. Traditional approaches of measuring cognitive load of surgeons utilise behavioural metrics to measure performance and surveys and questionnaires to collect reports of subjective experience. These have disadvantages such as sporadic data, occasionally intrusive methodologies, subjective or misleading self-reporting. In addition, traditional approaches use subjective metrics that cannot distinguish between skill levels. Functional neuroimaging data was collected using a high density, wireless NIRS device from sixteen surgeons (11 attending surgeons and 5 surgery resident) and 17 students while they performed two laparoscopic tasks (Peg transfer and String pass). Participant’s subjective mental load was assessed using the NASA-TLX survey. Machine learning approaches were used for predicting the subjective experience and skill levels. The Prefrontal cortex (PFC) activations were greater in students who reported higher-than-median task load, as measured by the NASA-TLX survey. However in the case of attending surgeons the opposite tendency was observed, namely higher activations in the lower v higher task loaded subjects. We found that response was greater in the left PFC of students particularly near the dorso- and ventrolateral areas. We quantified the ability of PFC activation to predict the differences in skill and task load using machine learning while focussing on the effects of NIRS channel separation distance on the results. Our results showed that the classification of skill level and subjective task load could be predicted based on PFC activation with an accuracy of nearly 90%. Our finding shows that there is sufficient information available in the optical signals to make accurate predictions about the surgeons’ subjective experiences and skill levels. The high accuracy of results is encouraging and suggest the integration of the strategy developed in this study as a promising approach to design automated, more accurate and objective evaluation methods.

Activation of Prefrontal Cortex in Process of Oral and Finger Shape Discrimination: fNIRS Study

Background

The differences in the brain activities of the insular and the visual association cortices have been reported between oral and manual stereognosis. However, these results were not conclusive because of the inherent differences in the task performance-related motor sequence conditions. We hypothesized that the involvement of the prefrontal cortex may be different between finger and oral shape discrimination. This study was conducted to clarify temporal changes in prefrontal activities occurring in the processes of oral and finger tactual shape discrimination using prefrontal functional near-infrared spectroscopy (fNIRS).

Methods

Six healthy right-handed males [aged 30.8 ± 8.2 years (mean ± SD)] were enrolled. Measurements of prefrontal activities were performed using a 22-channel fNIRS device (ETG-100, Hitachi Medical Co., Chiba, Japan) during experimental blocks that included resting state (REST), nonsense shape discrimination (SHAM), and shape discrimination (SHAPE).

Results

No significant difference was presented with regard to the number of correct answers during trials between oral and finger SHAPE discrimination. Additionally, a statistical difference for the prefrontal fNIRS activity between oral and finger shape discrimination was noted in CH 1. Finger SHAPE, as compared with SHAM, presented a temporally shifting onset and burst in the prefrontal activities from the frontopolar area (FPA) to the orbitofrontal cortex (OFC). In contrast, oral SHAPE as compared with SHAM was shown to be temporally overlapped in the onset and burst of the prefrontal activities in the dorsolateral prefrontal cortex (DLPFC)/FPA/OFC.

Conclusion

The prefrontal activities temporally shifting from the FPA to the OFC during SHAPE as compared with SHAM may suggest the segregated serial prefrontal processing from the manipulation of a target image to the decision making during the process of finger shape discrimination. In contrast, the temporally overlapped prefrontal activities of the DLPFC/FPA/OFC in the oral SHAPE block may suggest the parallel procession of the repetitive involvement of generation, manipulation, and decision making in order to form a reliable representation of target objects.

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Using Transcranial Near-Infrared Photobiomodulation With the Same Device

Transcranial near-infrared photobiomodulation (tNIR-PBM) can modulate physiological characteristics of the human brain, such as the cerebral blood flow and oxidative metabolism. Here, we investigated whether the performance of near-infrared spectroscopy (NIRS)-based brain-computer interfaces (BCIs) can be improved by tNIR-PBM applied to the prefrontal cortex with the same NIRS device. A total of 14 healthy individuals participated in the NIRS-based BCI study where the aim was to distinguish the mental arithmetic task from the idle state (IS) task either after tNIR-PBM or after sham stimulation, with the two experiments being conducted at least two days apart. The tNIR-PBM was applied by simply turning on the NIRS recording equipment for 20 min. To evaluate the degree of performance improvement obtained after tNIR-PBM, the average BCI classification accuracy obtained under the tNIR-PBM condition was compared with that obtained under the sham stimulation condition. The classification accuracy of NIRS-based BCI was significantly improved upon conduction of tNIR-PBM (82.74%) as compared to that in the sham stimulation condition (76.07%, p < 0.005). Thus, our results suggest that simply turning on the NIRS recording equipment before the BCI experiment can improve the performance of the NIRS-based BCI system.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain-computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

Decoding Three Different Preference Levels of Consumers Using Convolutional Neural Network: A Functional Near-Infrared Spectroscopy Study

This study decodes consumers' preference levels using a convolutional neural network (CNN) in neuromarketing. The classification accuracy in neuromarketing is a critical factor in evaluating the intentions of the consumers. Functional near-infrared spectroscopy (fNIRS) is utilized as a neuroimaging modality to measure the cerebral hemodynamic responses. In this study, a specific decoding structure, called CNN-based fNIRS-data analysis, was designed to achieve a high classification accuracy. Compared to other methods, the automated characteristics, constant training of the dataset, and learning efficiency of the proposed method are the main advantages. The experimental procedure required eight healthy participants (four female and four male) to view commercial advertisement videos of different durations (15, 30, and 60 s). The cerebral hemodynamic responses of the participants were measured. To compare the preference classification performances, CNN was utilized to extract the most common features, including the mean, peak, variance, kurtosis, and skewness. Considering three video durations, the average classification accuracies of 15, 30, and 60 s videos were 84.3, 87.9, and 86.4%, respectively. Among them, the classification accuracy of 87.9% for 30 s videos was the highest. The average classification accuracies of three preferences in females and males were 86.2 and 86.3%, respectively, showing no difference in each group. By comparing the classification performances in three different combinations (like vs. so-so, like vs. dislike, and so-so vs. dislike) between two groups, male participants were observed to have targeted preferences for commercial advertising, and the classification performance 88.4% between "like" vs. "dislike" out of three categories was the highest. Finally, pairwise classification performance are shown as follows: For female, 86.1% (like vs. so-so), 87.4% (like vs. dislike), 85.2% (so-so vs. dislike), and for male 85.7, 88.4, 85.1%, respectively.

Neural Correlates of Mental Rotation in Preschoolers With High or Low Working Memory Capacity: An fNIRS Study

This study explored the differentiated neural correlates of mental rotation (MR) in preschoolers with high and low working memory capacity using functional near-infrared spectroscopy (fNIRS). Altogether 38 Chinese preschoolers (M = 5.0 years, SD = 0.69 years) completed the Working Memory Capacity (WMC) test, the Mental Rotation (MR), and its Control tasks (without MR). They were divided into High-WMC (N₁ = 9) and Low-WMC (N₂ = 18) groups based on the WMC scores. The behavioral and fNIRS results indicated that: (1) there were no significant differences in MR task performance between the High-WMC (M_mr = 23.44, SD = 0.88) and Low-WMC group (M_mr = 23.67, SD = 0.59); (2) the Low-WMC group activated BA6, BA8, BA 9, and BA 44, whereas the High-WMC group activated BA8, BA10 and BA 44 during mental rotation; (3) significant differences were found in the activation of BA44 and BA9 between the High-WMC and Low-WMC groups during mental rotation; and (4) the High-WMC and Low-WMC groups differed significantly in the activation of BA 9 and BA10 during the control tasks, indicating that both areas might be responsible for the group differences in working memory.

Prefrontal Asymmetry BCI Neurofeedback Datasets

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Hybrid EEG-fNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD)

Brain-computer interface (BCI) multi-modal fusion has the potential to generate multiple commands in a highly reliable manner by alleviating the drawbacks associated with single modality. In the present work, a hybrid EEG-fNIRS BCI system—achieved through a fusion of concurrently recorded electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals—is used to overcome the limitations of uni-modality and to achieve higher tasks classification. Although the hybrid approach enhances the performance of the system, the improvements are still modest due to the lack of availability of computational approaches to fuse the two modalities. To overcome this, a novel approach is proposed using Multi-resolution singular value decomposition (MSVD) to achieve system- and feature-based fusion. The two approaches based up different features set are compared using the KNN and Tree classifiers. The results obtained through multiple datasets show that the proposed approach can effectively fuse both modalities with improvement in the classification accuracy.

Enhancing Classification Performance of fNIRS-BCI by Identifying Cortically Active Channels Using the z-Score Method

A state-of-the-art brain–computer interface (BCI) system includes brain signal acquisition, noise removal, channel selection, feature extraction, classification, and an application interface. In functional near-infrared spectroscopy-based BCI (fNIRS-BCI) channel selection may enhance classification performance by identifying suitable brain regions that contain brain activity. In this study, the z-score method for channel selection is proposed to improve fNIRS-BCI performance. The proposed method uses cross-correlation to match the similarity between desired and recorded brain activity signals, followed by forming a vector of each channel’s correlation coefficients’ maximum values. After that, the z-score is calculated for each value of that vector. A channel is selected based on a positive z-score value. The proposed method is applied to an open-access dataset containing mental arithmetic (MA) and motor imagery (MI) tasks for twenty-nine subjects. The proposed method is compared with the conventional t-value method and with no channel selected, i.e., using all channels. The z-score method yielded significantly improved (p < 0.0167) classification accuracies of 87.2 ± 7.0%, 88.4 ± 6.2%, and 88.1 ± 6.9% for left motor imagery (LMI) vs. rest, right motor imagery (RMI) vs. rest, and mental arithmetic (MA) vs. rest, respectively. The proposed method is also validated on an open-access database of 17 subjects, containing right-hand finger tapping (RFT), left-hand finger tapping (LFT), and dominant side foot tapping (FT) tasks.The study shows an enhanced performance of the z-score method over the t-value method as an advancement in efforts to improve state-of-the-art fNIRS-BCI systems’ performance.

Amplitude of fNIRS Resting-State Global Signal Is Related to EEG Vigilance Measures: A Simultaneous fNIRS and EEG Study

Recently, functional near-infrared spectroscopy (fNIRS) has been utilized to image the hemodynamic activities and connectivity in the human brain. With the advantage of economic efficiency, portability, and fewer physical constraints, fNIRS enables studying of the human brain at versatile environment and various body positions, including at bed side and during exercise, which complements the use of functional magnetic resonance imaging (fMRI). However, like fMRI, fNIRS imaging can be influenced by the presence of a strong global component. Yet, the nature of the global signal in fNIRS has not been established. In this study, we investigated the relationship between fNIRS global signal and electroencephalogram (EEG) vigilance using simultaneous recordings in resting healthy subjects in high-density and whole-head montage. In Experiment 1, data were acquired at supine, sitting, and standing positions. Results found that the factor of body positions significantly affected the amplitude of the resting-state fNIRS global signal, prominently in the frequency range of 0.05-0.1 Hz but not in the very low frequency range of less than 0.05 Hz. As a control, the task-induced fNIRS or EEG responses to auditory stimuli did not differ across body positions. However, EEG vigilance plays a modulatory role in the fNIRS signals in the frequency range of less than 0.05 Hz: resting-state sessions of low EEG vigilance measures are associated with high amplitudes of fNIRS global signals. Moreover, in Experiment 2, we further examined the epoch-to-epoch fluctuations in concurrent fNIRS and EEG data acquired from a separate group of subjects and found a negative temporal correlation between EEG vigilance measures and fNIRS global signal amplitudes. Our study for the first time revealed that vigilance as a neurophysiological factor modulates the resting-state dynamics of fNIRS, which have important implications for understanding and processing the noises in fNIRS signals.

Measuring dlPFC Signals to Predict the Success of Merchandising Elements at the Point-of-Sale - A fNIRS Approach

The (re-)launch of products is frequently accompanied by point-of-sale (PoS) marketing campaigns in order to foster sales. Predicting the success of these merchandising elements at the PoS on sales is of interest to research and practice, as the misinvestments that are based on the fragmented PoS literature are tremendous. Likewise, the predictive power of neuropsychological methods has been demonstrated in various research work. Nevertheless, the practical application of these neuropsychological methods is still limited. In order to foster the application of neuropsychological methods in research and practice, the current research work aims to explore, whether mobile functional near-infrared spectroscopy (fNIRS) - as a portable neuroimaging method - has the potential to predict the success of PoS merchandising elements by rendering significant neural signatures of brain regions of the dorsolateral prefrontal cortex (dlPFC), highlighting its potential to forecast shoppers' behaviour aka sales at the PoS. Building on previous research findings, the results of the given research work indicate that the neural signal of brain regions of the dlPFC, measured with mobile fNIRS, is able to predict actual sales associated with PoS merchandising elements, relying on the cortical relief effect. More precisely, the research findings support the hypothesis that the reduced neural activity of brain regions associated with the dlPFC can predict sales at the PoS, emphasising another crucial neural signature to predict shoppers' purchase behaviour, next to the frequently cited reward association system. The research findings offer an innovative perspective on how to design and evaluate PoS merchandising elements, indicating fruitful theoretical and practical implications.

A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers

Brain signals refer to the biometric information collected from the human brain. The research on brain signals aims to discover the underlying neurological or physical status of the individuals by signal decoding. The emerging deep learning techniques have improved the study of brain signals significantly in recent years. In this work, we first present a taxonomy of non-invasive brain signals and the basics of deep learning algorithms. Then, we provide a comprehensive survey of the frontiers of applying deep learning for non-invasive brain signals analysis, by summarizing a large number of recent publications. Moreover, upon the deep learning-powered brain signal studies, we report the potential real-world applications which benefit not only disabled people but also normal individuals. Finally, we discuss the opening challenges and future directions.

Performance Improvement for Detecting Brain Function Using fNIRS: A Multi-Distance Probe Configuration With PPL Method

To improve the spatial resolution of imaging and get more effective brain function information, a multi-distance probe configuration with three distances (28.2, 40, and 44.7 mm) and 52 channels is designed. At the same time, a data conversion method of modified Beer–Lambert law (MBLL) with partial pathlength (PPL) is proposed. In the experiment, three kinds of tasks, grip of left hand, grip of right hand, and rest, are performed with eight healthy subjects. First, with a typical single-distance probe configuration (30 mm, 24 channels), the feasibility of the proposed MBLL with PPL is preliminarily validated. Further, the characteristic of the proposed method is evaluated with the multi-distance probe configuration. Compared with MBLL with differential pathlength factor (DPF), the proposed MBLL with PPL is able to acquire more obvious concentration change and can achieve higher classification accuracy of the three tasks. Then, with the proposed method, the performance of the multi-distance probe configuration is discussed. Results show that, compared with a single distance, the combination of the three distances has better spatial resolution and could explore more accurate brain activation information. Besides, the classification accuracy of the three tasks obtained with the combination of three distances is higher than that of any combination of two distances. Also, with the combination of the three distances, the two-class classification between different tasks is carried out. Both theory and experimental results demonstrate that, using multi-distance probe configuration and the MBLL with PPL method, the performance of brain function detected by NIRS can be improved.

Wearable hip-assist robot modulates cortical activation during gait in stroke patients: a functional near-infrared spectroscopy study

Background

Gait dysfunction is common in post-stroke patients as a result of impairment in cerebral gait mechanism. Powered robotic exoskeletons are promising tools to maximize neural recovery by delivering repetitive walking practice.

Objectives

The purpose of this study was to investigate the modulating effect of the Gait Enhancing and Motivating System-Hip (GEMS-H) on cortical activation during gait in patients with chronic stroke. Methods. Twenty chronic stroke patients performed treadmill walking at a self-selected speed either with assistance of GEMS-H (GEMS-H) or without assistance of GEMS-H (NoGEMS-H). Changes in oxygenated hemoglobin (oxyHb) concentration in the bilateral primary sensorimotor cortex (SMC), premotor cortices (PMC), supplemental motor areas (SMA), and prefrontal cortices (PFC) were recorded using functional near infrared spectroscopy.

Results

Walking with the GEMS-H promoted symmetrical SMC activation, with more activation in the affected hemisphere than in NoGEMS-H conditions. GEMS-H also decreased oxyHb concentration in the late phase over the ipsilesional SMC and bilateral SMA (P < 0.05).

Conclusions

The results of the present study reveal that the GEMS-H promoted more SMC activation and a balanced activation pattern that helped to restore gait function. Less activation in the late phase over SMC and SMA during gait with GEMS-H indicates that GEMS-H reduces the cortical participation of stroke gait by producing rhythmic hip flexion and extension movement and allows a more coordinate and efficient gait patterns.

Trial registration NCT03048968. Registered 06 Feb 2017

Therapeutic benefits of music-based synchronous finger tapping in Parkinson's disease-an fNIRS study protocol for randomized controlled trial in Dalian, China

BACKGROUND

Music therapy improves neuronal activity and connectivity of healthy persons and patients with clinical symptoms of neurological diseases like Parkinson's disease, Alzheimer's disease, and major depression. Despite the plethora of publications that have reported the positive effects of music interventions, little is known about how music improves neuronal activity and connectivity in afflicted patients.

METHODS

For patients suffering from Parkinson's disease (PD), we propose a daily 25-min music-based synchronous finger tapping (SFT) intervention for 8 weeks. Eligible participants with PD are split into two groups: an intervention group and a control arm. In addition, a third cohort of healthy controls will be recruited. Assessment of finger tapping performances, the Unified Parkinson's Disease Rating Scale (UPDRS), an n-back test, the Montreal Cognitive Assessment (MoCA), as well as oxygenated hemoglobin (HbO2), deoxygenated hemoglobin (HbR), and total hemoglobin activation collected by functional near-infrared spectroscopy (fNIRS) are measured at baseline, week 4 (during), week 8 (post), and week 12 (retention) of the study. Data collected from the two PD groups are compared to baseline performances from healthy controls.

DISCUSSION

This exploratory prospective trial study investigates the cortical neuronal activity and therapeutic effects associated with an auditory external cue used to induce automatic and implicit synchronous finger tapping in patients diagnosed with PD. The extent to which the intervention is effective may be dependent on the severity of the disease. The study's findings are used to inform larger clinical studies for optimization and further exploration of the therapeutic effects of movement-based music therapy on neural activity in neurological diseases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04212897 . Registered on December 30, 2019. The participant recruitment and study protocol have received ethical approval from the First Affiliated Hospital of Dalian Medical University. The hospital Protocol Record number is PJ-KY-2019-123. The protocol was named "fNIRS Studies of Music Intervention of Parkinson's Disease." The current protocol is version 1.1, revised on September 1, 2020.

Transcranial photobiomodulation-induced changes in human brain functional connectivity and network metrics mapped by whole-head functional near-infrared spectroscopy in vivo

Transcranial photobiomodulation (tPBM) with near-infrared light on the human head has been shown to enhance human cognition. In this study, tPBM-induced effects on resting state brain networks were investigated using 111-channel functional near-infrared spectroscopy over the whole head. Measurements were collected with and without 8-minute tPBM in 19 adults. Functional connectivity (FC) and brain network metrics were quantified using Pearson's correlation coefficients and graph theory analysis (GTA), respectively, for the periods of pre-, during, and post-tPBM. Our results revealed that tPBM (1) enhanced information processing speed and efficiency of the brain network, and (2) increased FC significantly in the frontal-parietal network, shedding light on a better understanding of tPBM effects on brain networks.

A Unified Analytical Framework With Multiple fNIRS Features for Mental Workload Assessment in the Prefrontal Cortex

Knowing the actual level of mental workload is important to ensure the efficacy of brain-computer interface (BCI) based cognitive training. Extracting signals from limited area of a brain region might not reveal the actual information. In this study, a functional near-infrared spectroscopy (fNIRS) device equipped with multi-channel and multi-distance measurement capability was employed for the development of an analytical framework to assess mental workload in the prefrontal cortex (PFC). In addition to the conventional features, e.g. hemodynamic slope, we introduced a new feature - deep contribution ratio which is the proportion of cerebral hemodynamics to the fNIRS signals. Multiple sets of features were examined by a simple logical operator to suppress the false detection rate in identifying the activated channels. Using the number of activated channels as input to a linear support vector machine (SVM), the performance of the proposed analytical framework was assessed in classifying three levels of mental workload. The best set of features involves the combination of hemodynamic slope and deep contribution ratio, where the identified number of activated channels returned an average accuracy of 80.6% in predicting mental workload, compared to a single conventional feature (accuracy: 59.8%). This suggests the feasibility of the proposed analytical framework with multiple features as a means towards a more accurate assessment of mental workload in fNIRS-based BCI applications.

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

The goal of this study was to develop and validate a hybrid brain-computer interface (BCI) system for home automation control. Over the past decade, BCIs represent a promising possibility in the field of medical (e.g., neuronal rehabilitation), educational, mind reading, and remote communication. However, BCI is still difficult to use in daily life because of the challenges of the unfriendly head device, lower classification accuracy, high cost, and complex operation. In this study, we propose a hybrid BCI system for home automation control with two brain signals acquiring electrodes and simple tasks, which only requires the subject to focus on the stimulus and eye blink. The stimulus is utilized to select commands by generating steady-state visually evoked potential (SSVEP). The single eye blinks (i.e., confirm the selection) and double eye blinks (i.e., deny and re-selection) are employed to calibrate the SSVEP command. Besides that, the short-time Fourier transform and convolution neural network algorithms are utilized for feature extraction and classification, respectively. The results show that the proposed system could provide 38 control commands with a 2 s time window and a good accuracy (i.e., 96.92%) using one bipolar electroencephalogram (EEG) channel. This work presents a novel BCI approach for the home automation application based on SSVEP and eye blink signals, which could be useful for the disabled. In addition, the provided strategy of this study—a friendly channel configuration (i.e., one bipolar EEG channel), high accuracy, multiple commands, and short response time—might also offer a reference for the other BCI controlled applications.

Recent Developments in Instrumentation of Functional Near-Infrared Spectroscopy Systems

In the last three decades, the development and steady improvement of various optical technologies at the near-infrared region of the electromagnetic spectrum has inspired a large number of scientists around the world to design and develop functional near-infrared spectroscopy (fNIRS) systems for various medical applications. This has been driven further by the availability of new sources and detectors that support very compact and wearable system designs. In this article, we review fNIRS systems from the instrumentation point of view, discussing the associated challenges and state-of-the-art approaches. In the beginning, the fundamentals of fNIRS systems as well as light-tissue interaction at NIR are briefly introduced. After that, we present the basics of NIR systems instrumentation. Next, the recent development of continuous-wave, frequency-domain, and time-domain fNIRS systems are discussed. Finally, we provide a summary of these three modalities and an outlook into the future of fNIRS technology.