Real-time feature extraction of P300 component using adaptive nonlinear principal component analysis

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Deception is an inevitable occurrence in daily life. Various methods have been used to understand the mechanisms underlying brain deception. Moreover, numerous efforts have been undertaken to detect deception and truth-telling. Functional near-infrared spectroscopy (fNIRS) has great potential for neurological applications compared with other state-of-the-art methods. Therefore, an fNIRS-based spontaneous lie detection model was used in the present study. We interviewed 10 healthy subjects to identify deception using the fNIRS system. A card game frequently referred to as a bluff or cheat was introduced. This game was selected because its rules are ideal for testing our hypotheses. The optical probe of the fNIRS was placed on the subject's forehead, and we acquired optical density signals, which were then converted into oxy-hemoglobin and deoxy-hemoglobin signals using the Modified Beer-Lambert law. The oxy-hemoglobin signal was preprocessed to eliminate noise. In this study, we proposed three artificial neural networks inspired by deep learning models, including AlexNet, ResNet, and GoogleNet, to classify deception and truth-telling. The proposed models achieved accuracies of 88.5%, 88.0%, and 90.0%, respectively. These proposed models were compared with other classification models, including k-nearest neighbor, linear support vector machines (SVM), quadratic SVM, cubic SVM, simple decision trees, and complex decision trees. These comparisons showed that the proposed models performed better than the other state-of-the-art methods.

Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography

Surprise and social influence are linked through several neuropsychological mechanisms. By garnering attention, causing arousal, and motivating engagement, surprise provides a context for effective or durable social influence. Attention to a surprising event motivates the formation of an explanation or updating of models, while high arousal experiences due to surprise promote memory formation. They both encourage engagement with the surprising event through efforts aimed at understanding the situation. By affecting the behavior of the individual or a social group via setting an attractive engagement context, surprise plays an important role in shaping personal and social change. Surprise is an outcome of the brain’s function in constantly anticipating the future of sensory inputs based on past experiences. When new sensory data is different from the brain’s predictions shaped by recent trends, distinct neural signals are generated to report this surprise. As a quantitative approach to modeling the generation of brain surprise, input stimuli containing surprising elements are employed in experiments such as oddball tasks during which brain activity is recorded. Although surprise has been well characterized in many studies, an information-theoretical model to describe and predict the surprise level of an external stimulus in the recorded MEG data has not been reported to date, and setting forth such a model is the main objective of this paper. Through mining trial-by-trial MEG data in an oddball task according to theoretical definitions of surprise, the proposed surprise decoding model employs the entire epoch of the brain response to a stimulus to measure surprise and assesses which collection of temporal/spatial components in the recorded data can provide optimal power for describing the brain’s surprise. We considered three different theoretical formulations for surprise assuming the brain acts as an ideal observer that calculates transition probabilities to estimate the generative distribution of the input. We found that middle temporal components and the right and left fronto-central regions offer the strongest power for decoding surprise. Our findings provide a practical and rigorous method for measuring the brain’s surprise, which can be employed in conjunction with behavioral data to evaluate the interactive and social effects of surprising events.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface

A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Quantitative Assessment of Resting-State for Mild Cognitive Impairment Detection: A Functional Near-Infrared Spectroscopy and Deep Learning Approach

BACKGROUND

Mild cognitive impairment (MCI) is considered a prodromal stage of Alzheimer's disease. Early diagnosis of MCI can allow for treatment to improve cognitive function and reduce modifiable risk factors.

OBJECTIVE

This study aims to investigate the feasibility of individual MCI detection from healthy control (HC) using a minimum duration of resting-state functional near-infrared spectroscopy (fNIRS) signals.

METHODS

In this study, nine different measurement durations (i.e., 30, 60, 90, 120, 150, 180, 210, 240, and 270 s) were evaluated for MCI detection via the graph theory analysis and traditional machine learning approach, such as linear discriminant analysis, support vector machine, and K-nearest neighbor algorithms. Moreover, feature representation- and classification-based transfer learning (TL) methods were applied to identify MCI from HC through the input of connectivity maps with 30 and 90 s duration.

RESULTS

There was no significant difference among the nine various time windows in the machine learning and graph theory analysis. The feature representation-based TL showed improved accuracy in both 30 and 90 s cases (i.e., 30 s: 81.27% and 90 s: 76.73%). Notably, the classification-based TL method achieved the highest accuracy of 95.81% using the pre-trained convolutional neural network (CNN) model with the 30 s interval functional connectivity map input.

CONCLUSION

The results indicate that a 30 s measurement of the resting-state with fNIRS could be used to detect MCI. Moreover, the combination of neuroimaging (e.g., functional connectivity maps) and deep learning methods (e.g., CNN and TL) can be considered as novel biomarkers for clinical computer-assisted MCI diagnosis.

Learning Discriminative Spatiospectral Features of ERPs for Accurate Brain-Computer Interfaces

Constructing accurate predictive models is at the heart of brain-computer interfaces (BCIs) because these models can ultimately translate brain activities into communication and control commands. The majority of the previous work in BCI use spatial, temporal, or spatiotemporal features of event-related potentials (ERPs). In this study, we examined the discriminatory effect of their spatiospectral features to capture the most relevant set of neural activities from electroencephalographic recordings that represent users' mental intent. In this regard, we model ERP waveforms using a sum of sinusoids with unknown amplitudes, frequencies, and phases. The effect of this signal modeling step is to represent high-dimensional ERP waveforms in a substantially lower dimensionality space, which includes their dominant power spectral contents. We found that the most discriminative frequencies for accurate decoding of visual attention modulated ERPs lie in a spectral range less than 6.4 Hz. This was empirically verified by treating dominant frequency contents of ERP waveforms as feature vectors in the state-of-the-art machine learning techniques used herein. The constructed predictive models achieved remarkable performance, which for some subjects was as high as 94% as measured by the area under curve. Using these spectral contents, we further studied the discriminatory effect of each channel and proposed an efficient strategy to choose subject-specific subsets of channels that generally led to classifiers with comparable performance.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

Classification of electroencephalogram signals using wavelet-CSP and projection extreme learning machine

Brain-computer interface (BCI) systems establish a direct communication channel from the brain to an output device. As the basis of BCIs, recognizing motor imagery activities poses a considerable challenge to signal processing due to the complex and non-stationary characteristics. This paper introduces an optimal and intelligent method for motor imagery BCIs. Because of the robustness to noise, wavelet packet decomposition and common spatial pattern (CSP) methods were implemented to reduce the dimensions of preprocessed signals. And a novel and efficient classifier projection extreme learning machine (PELM) was employed to recognize the labels of electroencephalogram signals. Experiments have been performed on the BCI Competition Dataset to demonstrate the superiority of wavelet-CSP in BCI and the outperformance of the PELM-based method. Results show that the average recognition rate of PELM approaches approximately 70%, while the optimal rate of other methods is 72%, whose training time and classification time are relatively longer as 11.00 ms and 11.66 ms, respectively, compared with 4.75 ms and 4.87 ms obtained by using the proposed BCI system.

Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Robust Adaptive Synchronization of Ring Configured Uncertain Chaotic FitzHugh-Nagumo Neurons under Direction-Dependent Coupling

This paper exploits the dynamical modeling, behavior analysis, and synchronization of a network of four different FitzHugh–Nagumo (FHN) neurons with unknown parameters linked in a ring configuration under direction-dependent coupling. The main purpose is to investigate a robust adaptive control law for the synchronization of uncertain and perturbed neurons, communicating in a medium of bidirectional coupling. The neurons are assumed to be different and interconnected in a ring structure. The strength of the gap junctions is taken to be different for each link in the network, owing to the inter-neuronal coupling medium properties. Robust adaptive control mechanism based on Lyapunov stability analysis is employed and theoretical criteria are derived to realize the synchronization of the network of four FHN neurons in a ring form with unknown parameters under direction-dependent coupling and disturbances. The proposed scheme for synchronization of dissimilar neurons, under external electrical stimuli, coupled in a ring communication topology, having all parameters unknown, and subject to directional coupling medium and perturbations, is addressed for the first time as per our knowledge. To demonstrate the efficacy of the proposed strategy, simulation results are provided.

Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review

For those individuals with upper-extremity amputation, a daily normal living activity is no longer possible or it requires additional effort and time. With the aim of restoring their sensory and motor functions, theoretical and technological investigations have been carried out in the field of neuroprosthetic systems. For transmission of sensory feedback, several interfacing modalities including indirect (non-invasive), direct-to-peripheral-nerve (invasive), and cortical stimulation have been applied. Peripheral nerve interfaces demonstrate an edge over the cortical interfaces due to the sensitivity in attaining cortical brain signals. The peripheral nerve interfaces are highly dependent on interface designs and are required to be biocompatible with the nerves to achieve prolonged stability and longevity. Another criterion is the selection of nerves that allows minimal invasiveness and damages as well as high selectivity for a large number of nerve fascicles. In this paper, we review the nerve-machine interface modalities noted above with more focus on peripheral nerve interfaces, which are responsible for provision of sensory feedback. The invasive interfaces for recording and stimulation of electro-neurographic signals include intra-fascicular, regenerative-type interfaces that provide multiple contact channels to a group of axons inside the nerve and the extra-neural-cuff-type interfaces that enable interaction with many axons around the periphery of the nerve. Section Current Prosthetic Technology summarizes the advancements made to date in the field of neuroprosthetics toward the achievement of a bidirectional nerve-machine interface with more focus on sensory feedback. In the Discussion section, the authors propose a hybrid interface technique for achieving better selectivity and long-term stability using the available nerve interfacing techniques.

Decreased functional connectivity and disrupted neural network in the prefrontal cortex of affective disorders: A resting-state fNIRS study

BACKGROUND

Affective disorders (AD) have been conceptualized as neural network-level diseases. In this study, we utilized functional near infrared spectroscopy (fNIRS) to investigate the spontaneous hemodynamic activities in the prefrontal cortex (PFC) of the AD patients with or without medications.

METHODS

42 optical channels were applied to cover the superior frontal gyrus (SFG), middle frontal gyrus (MFG), and inferior frontal gyrus (IFG), which constitute one of the most important affective networks of the brain. We performed resting-state measurements on 28 patients who were diagnosed as having AD and 30 healthy controls (HC). Raw fNIRS data were preprocessed with independent component analysis (ICA) and a band-pass filter to remove artifacts and physiological noise.

RESULTS

By systematically analyzing the intra-regional, intrahemispheric, and interhemispheric connectivities based on the spontaneous oscillations of Δ[HbO], our results indicated that patients with AD exhibited significantly reduced intra-regional and symmetrically interhemispheric connectivities in the PFC when compared to HC. More specifically, relative to HC, AD patients showed significantly lower locally functional connectivity in the right IFG, and poor long-distance connectivity between bilateral IFG. In addition, AD patients without medication presented more disrupted cortical organizations in the PFC, and the severity of self-reported symptoms of depression was negatively correlated with the strength of intra-regional and symmetrically interhemispheric connectivity in the PFC.

LIMITATIONS

Regarding the measuring technique, fNIRS has restricted measurement depth and spatial resolution. During the study, the subgroups of AD, such as major depressive disorder, bipolar, comorbidity, or non-comorbidity, dosage of psychotropic drugs, as well as different types of pharmacological responses were not distinguished and systematically compared. Furthermore, due to the limitation of the research design, it was still not very clear how pharmacological treatment affected the resting state cortical organization of the prefrontal lobe, and the degree of the effect in patients with AD.

CONCLUSION

These results strongly supported that RSFC measured by fNIRS could be a useful and powerful way of delineating the neuropathology of AD.

The Role of Visual Noise in Influencing Mental Load and Fatigue in a Steady-State Motion Visual Evoked Potential-Based Brain-Computer Interface

As a spatial selective attention-based brain-computer interface (BCI) paradigm, steady-state visual evoked potential (SSVEP) BCI has the advantages of high information transfer rate, high tolerance to artifacts, and robust performance across users. However, its benefits come at the cost of mental load and fatigue occurring in the concentration on the visual stimuli. Noise, as a ubiquitous random perturbation with the power of randomness, may be exploited by the human visual system to enhance higher-level brain functions. In this study, a novel steady-state motion visual evoked potential (SSMVEP, i.e., one kind of SSVEP)-based BCI paradigm with spatiotemporal visual noise was used to investigate the influence of noise on the compensation of mental load and fatigue deterioration during prolonged attention tasks. Changes in α, θ, θ + α powers, θ/α ratio, and electroencephalography (EEG) properties of amplitude, signal-to-noise ratio (SNR), and online accuracy, were used to evaluate mental load and fatigue. We showed that presenting a moderate visual noise to participants could reliably alleviate the mental load and fatigue during online operation of visual BCI that places demands on the attentional processes. This demonstrated that noise could provide a superior solution to the implementation of visual attention controlling-based BCI applications.

Hybrid Brain-Computer Interface Techniques for Improved Classification Accuracy and Increased Number of Commands: A Review

In this article, non-invasive hybrid brain-computer interface (hBCI) technologies for improving classification accuracy and increasing the number of commands are reviewed. Hybridization combining more than two modalities is a new trend in brain imaging and prosthesis control. Electroencephalography (EEG), due to its easy use and fast temporal resolution, is most widely utilized in combination with other brain/non-brain signal acquisition modalities, for instance, functional near infrared spectroscopy (fNIRS), electromyography (EMG), electrooculography (EOG), and eye tracker. Three main purposes of hybridization are to increase the number of control commands, improve classification accuracy and reduce the signal detection time. Currently, such combinations of EEG + fNIRS and EEG + EOG are most commonly employed. Four principal components (i.e., hardware, paradigm, classifiers, and features) relevant to accuracy improvement are discussed. In the case of brain signals, motor imagination/movement tasks are combined with cognitive tasks to increase active brain-computer interface (BCI) accuracy. Active and reactive tasks sometimes are combined: motor imagination with steady-state evoked visual potentials (SSVEP) and motor imagination with P300. In the case of reactive tasks, SSVEP is most widely combined with P300 to increase the number of commands. Passive BCIs, however, are rare. After discussing the hardware and strategies involved in the development of hBCI, the second part examines the approaches used to increase the number of control commands and to enhance classification accuracy. The future prospects and the extension of hBCI in real-time applications for daily life scenarios are provided.

Classification of somatosensory cortex activities using fNIRS

The ability of the somatosensory cortex in differentiating various tactile sensations is very important for a person to perceive the surrounding environment. In this study, we utilize a lab-made multi-channel functional near-infrared spectroscopy (fNIRS) to discriminate the hemodynamic responses (HRs) of four different tactile stimulations (handshake, ball grasp, poking, and cold temperature) applied to the right hand of eight healthy male subjects. The activated brain areas per stimulation are identified with the t-values between the measured data and the desired hemodynamic response function. Linear discriminant analysis is utilized to classify the acquired data into four classes based on three features (mean, peak value, and skewness) of the associated oxy-hemoglobin (HbO) signals. The HRs evoked by the handshake and poking stimulations showed higher peak values in HbO than the ball grasp and cold temperature stimulations. For comparison purposes, additional two-class classifications of poking vs. temperature and handshake vs. ball grasp were performed. The attained classification accuracies were higher than the corresponding chance levels. Our results indicate that fNIRS can be used as an objective measure discriminating different tactile stimulations from the somatosensory cortex of human brain.

Hybrid EEG-fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control

In this paper, a hybrid electroencephalography–functional near-infrared spectroscopy (EEG–fNIRS) scheme to decode eight active brain commands from the frontal brain region for brain–computer interface is presented. A total of eight commands are decoded by fNIRS, as positioned on the prefrontal cortex, and by EEG, around the frontal, parietal, and visual cortices. Mental arithmetic, mental counting, mental rotation, and word formation tasks are decoded with fNIRS, in which the selected features for classification and command generation are the peak, minimum, and mean ΔHbO values within a 2-s moving window. In the case of EEG, two eyeblinks, three eyeblinks, and eye movement in the up/down and left/right directions are used for four-command generation. The features in this case are the number of peaks and the mean of the EEG signal during 1 s window. We tested the generated commands on a quadcopter in an open space. An average accuracy of 75.6% was achieved with fNIRS for four-command decoding and 86% with EEG for another four-command decoding. The testing results show the possibility of controlling a quadcopter online and in real-time using eight commands from the prefrontal and frontal cortices via the proposed hybrid EEG–fNIRS interface.

Hybrid brain-computer interface for biomedical cyber-physical system application using wireless embedded EEG systems

Background

One of the key challenges of the biomedical cyber-physical system is to combine cognitive neuroscience with the integration of physical systems to assist people with disabilities. Electroencephalography (EEG) has been explored as a non-invasive method of providing assistive technology by using brain electrical signals.

Methods

This paper presents a unique prototype of a hybrid brain computer interface (BCI) which senses a combination classification of mental task, steady state visual evoked potential (SSVEP) and eyes closed detection using only two EEG channels. In addition, a microcontroller based head-mounted battery-operated wireless EEG sensor combined with a separate embedded system is used to enhance portability, convenience and cost effectiveness. This experiment has been conducted with five healthy participants and five patients with tetraplegia.

Results

Generally, the results show comparable classification accuracies between healthy subjects and tetraplegia patients. For the offline artificial neural network classification for the target group of patients with tetraplegia, the hybrid BCI system combines three mental tasks, three SSVEP frequencies and eyes closed, with average classification accuracy at 74% and average information transfer rate (ITR) of the system of 27 bits/min. For the real-time testing of the intentional signal on patients with tetraplegia, the average success rate of detection is 70% and the speed of detection varies from 2 to 4 s.

Robust estimation of event-related potentials via particle filter

BACKGROUND AND OBJECTIVE

In clinical examinations and brain-computer interface (BCI) research, a short electroencephalogram (EEG) measurement time is ideal. The use of event-related potentials (ERPs) relies on both estimation accuracy and processing time. We tested a particle filter that uses a large number of particles to construct a probability distribution.

METHODS

We constructed a simple model for recording EEG comprising three components: ERPs approximated via a trend model, background waves constructed via an autoregressive model, and noise. We evaluated the performance of the particle filter based on mean squared error (MSE), P300 peak amplitude, and latency. We then compared our filter with the Kalman filter and a conventional simple averaging method. To confirm the efficacy of the filter, we used it to estimate ERP elicited by a P300 BCI speller.

RESULTS

A 400-particle filter produced the best MSE. We found that the merit of the filter increased when the original waveform already had a low signal-to-noise ratio (SNR) (i.e., the power ratio between ERP and background EEG). We calculated the amount of averaging necessary after applying a particle filter that produced a result equivalent to that associated with conventional averaging, and determined that the particle filter yielded a maximum 42.8% reduction in measurement time. The particle filter performed better than both the Kalman filter and conventional averaging for a low SNR in terms of both MSE and P300 peak amplitude and latency. For EEG data produced by the P300 speller, we were able to use our filter to obtain ERP waveforms that were stable compared with averages produced by a conventional averaging method, irrespective of the amount of averaging.

CONCLUSIONS

We confirmed that particle filters are efficacious in reducing the measurement time required during simulations with a low SNR. Additionally, particle filters can perform robust ERP estimation for EEG data produced via a P300 speller.

Passive BCI based on drowsiness detection: an fNIRS study

We use functional near-infrared spectroscopy (fNIRS) to discriminate the alert and drowsy states for a passive brain-computer interface (BCI). The passive brain signals for the drowsy state are acquired from the prefrontal and dorsolateral prefrontal cortex. The experiment is performed on 13 healthy subjects using a driving simulator, and their brain activity is recorded using a continuous-wave fNIRS system. Linear discriminant analysis (LDA) is employed for training and testing, using the data from the prefrontal, left- and right-dorsolateral prefrontal regions. For classification, eight features are tested: mean oxyhemoglobin, mean deoxyhemoglobin, skewness, kurtosis, signal slope, number of peaks, sum of peaks, and signal peak, in 0~5, 0~10, and 0~15 second time windows, respectively. The results show that the best performance for classification is achieved using mean oxyhemoglobin, the signal peak, and the sum of peaks as features. The average accuracies in the right dorsolateral prefrontal cortex (83.1, 83.4 and 84.9% in the 0~5, 0~10 and 0~15 second time windows, respectively) show that the proposed method has an effective utility for detection of drowsiness for a passive BCI.

Single-trial lie detection using a combined fNIRS-polygraph system

Deception is a human behavior that many people experience in daily life. It involves complex neuronal activities in addition to several physiological changes in the body. A polygraph, which can measure some of the physiological responses from the body, has been widely employed in lie-detection. Many researchers, however, believe that lie detection can become more precise if the neuronal changes that occur in the process of deception can be isolated and measured. In this study, we combine both measures (i.e., physiological and neuronal changes) for enhanced lie-detection. Specifically, to investigate the deception-related hemodynamic response, functional near-infrared spectroscopy (fNIRS) is applied at the prefrontal cortex besides a commercially available polygraph system. A mock crime scenario with a single-trial stimulus is set up as a deception protocol. The acquired data are classified into “true” and “lie” classes based on the fNIRS-based hemoglobin-concentration changes and polygraph-based physiological signal changes. Linear discriminant analysis is utilized as a classifier. The results indicate that the combined fNIRS-polygraph system delivers much higher classification accuracy than that of a singular system. This study demonstrates a plausible solution toward single-trial lie-detection by combining fNIRS and the polygraph.

fNIRS-based brain-computer interfaces: a review

A brain-computer interface (BCI) is a communication system that allows the use of brain activity to control computers or other external devices. It can, by bypassing the peripheral nervous system, provide a means of communication for people suffering from severe motor disabilities or in a persistent vegetative state. In this paper, brain-signal generation tasks, noise removal methods, feature extraction/selection schemes, and classification techniques for fNIRS-based BCI are reviewed. The most common brain areas for fNIRS BCI are the primary motor cortex and the prefrontal cortex. In relation to the motor cortex, motor imagery tasks were preferred to motor execution tasks since possible proprioceptive feedback could be avoided. In relation to the prefrontal cortex, fNIRS showed a significant advantage due to no hair in detecting the cognitive tasks like mental arithmetic, music imagery, emotion induction, etc. In removing physiological noise in fNIRS data, band-pass filtering was mostly used. However, more advanced techniques like adaptive filtering, independent component analysis (ICA), multi optodes arrangement, etc. are being pursued to overcome the problem that a band-pass filter cannot be used when both brain and physiological signals occur within a close band. In extracting features related to the desired brain signal, the mean, variance, peak value, slope, skewness, and kurtosis of the noised-removed hemodynamic response were used. For classification, the linear discriminant analysis method provided simple but good performance among others: support vector machine (SVM), hidden Markov model (HMM), artificial neural network, etc. fNIRS will be more widely used to monitor the occurrence of neuro-plasticity after neuro-rehabilitation and neuro-stimulation. Technical breakthroughs in the future are expected via bundled-type probes, hybrid EEG-fNIRS BCI, and through the detection of initial dips.

Lateralization of music processing with noises in the auditory cortex: an fNIRS study

The present study is to determine the effects of background noise on the hemispheric lateralization in music processing by exposing 14 subjects to four different auditory environments: music segments only, noise segments only, music + noise segments, and the entire music interfered by noise segments. The hemodynamic responses in both hemispheres caused by the perception of music in 10 different conditions were measured using functional near-infrared spectroscopy. As a feature to distinguish stimulus-evoked hemodynamics, the difference between the mean and the minimum value of the hemodynamic response for a given stimulus was used. The right-hemispheric lateralization in music processing was about 75% (instead of continuous music, only music segments were heard). If the stimuli were only noises, the lateralization was about 65%. But, if the music was mixed with noises, the right-hemispheric lateralization has increased. Particularly, if the noise was a little bit lower than the music (i.e., music level 10~15%, noise level 10%), the entire subjects showed the right-hemispheric lateralization: This is due to the subjects' effort to hear the music in the presence of noises. However, too much noise has reduced the subjects' discerning efforts.

A novel EOG/EEG hybrid human-machine interface adopting eye movements and ERPs: application to robot control

This study presents a novel human-machine interface (HMI) based on both electrooculography (EOG) and electroencephalography (EEG). This hybrid interface works in two modes: an EOG mode recognizes eye movements such as blinks, and an EEG mode detects event related potentials (ERPs) like P300. While both eye movements and ERPs have been separately used for implementing assistive interfaces, which help patients with motor disabilities in performing daily tasks, the proposed hybrid interface integrates them together. In this way, both the eye movements and ERPs complement each other. Therefore, it can provide a better efficiency and a wider scope of application. In this study, we design a threshold algorithm that can recognize four kinds of eye movements including blink, wink, gaze, and frown. In addition, an oddball paradigm with stimuli of inverted faces is used to evoke multiple ERP components including P300, N170, and VPP. To verify the effectiveness of the proposed system, two different online experiments are carried out. One is to control a multifunctional humanoid robot, and the other is to control four mobile robots. In both experiments, the subjects can complete tasks effectively by using the proposed interface, whereas the best completion time is relatively short and very close to the one operated by hand.

Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface

The hybrid brain-computer interface (BCI)'s multimodal technology enables precision brain-signal classification that can be used in the formulation of control commands. In the present study, an experimental hybrid near-infrared spectroscopy-electroencephalography (NIRS-EEG) technique was used to extract and decode four different types of brain signals. The NIRS setup was positioned over the prefrontal brain region, and the EEG over the left and right motor cortex regions. Twelve subjects participating in the experiment were shown four direction symbols, namely, “forward,” “backward,” “left,” and “right.” The control commands for forward and backward movement were estimated by performing arithmetic mental tasks related to oxy-hemoglobin (HbO) changes. The left and right directions commands were associated with right and left hand tapping, respectively. The high classification accuracies achieved showed that the four different control signals can be accurately estimated using the hybrid NIRS-EEG technology.

A low-cost EEG system-based hybrid brain-computer interface for humanoid robot navigation and recognition

This paper describes a hybrid brain-computer interface (BCI) technique that combines the P300 potential, the steady state visually evoked potential (SSVEP), and event related de-synchronization (ERD) to solve a complicated multi-task problem consisting of humanoid robot navigation and control along with object recognition using a low-cost BCI system. Our approach enables subjects to control the navigation and exploration of a humanoid robot and recognize a desired object among candidates. This study aims to demonstrate the possibility of a hybrid BCI based on a low-cost system for a realistic and complex task. It also shows that the use of a simple image processing technique, combined with BCI, can further aid in making these complex tasks simpler. An experimental scenario is proposed in which a subject remotely controls a humanoid robot in a properly sized maze. The subject sees what the surrogate robot sees through visual feedback and can navigate the surrogate robot. While navigating, the robot encounters objects located in the maze. It then recognizes if the encountered object is of interest to the subject. The subject communicates with the robot through SSVEP and ERD-based BCIs to navigate and explore with the robot, and P300-based BCI to allow the surrogate robot recognize their favorites. Using several evaluation metrics, the performances of five subjects navigating the robot were quite comparable to manual keyboard control. During object recognition mode, favorite objects were successfully selected from two to four choices. Subjects conducted humanoid navigation and recognition tasks as if they embodied the robot. Analysis of the data supports the potential usefulness of the proposed hybrid BCI system for extended applications. This work presents an important implication for the future work that a hybridization of simple BCI protocols provide extended controllability to carry out complicated tasks even with a low-cost system.

Noise reduction in functional near-infrared spectroscopy signals by independent component analysis

Functional near-infrared spectroscopy (fNIRS) is used to detect concentration changes of oxy-hemoglobin and deoxy-hemoglobin in the human brain. The main difficulty entailed in the analysis of fNIRS signals is the fact that the hemodynamic response to a specific neuronal activation is contaminated by physiological and instrument noises, motion artifacts, and other interferences. This paper proposes independent component analysis (ICA) as a means of identifying the original hemodynamic response in the presence of noises. The original hemodynamic response was reconstructed using the primary independent component (IC) and other, less-weighting-coefficient ICs. In order to generate experimental brain stimuli, arithmetic tasks were administered to eight volunteer subjects. The t-value of the reconstructed hemodynamic response was improved by using the ICs found in the measured data. The best t-value out of 16 low-pass-filtered signals was 37, and that of the reconstructed one was 51. Also, the average t-value of the eight subjects' reconstructed signals was 40, whereas that of all of their low-pass-filtered signals was only 20. Overall, the results showed the applicability of the ICA-based method to noise-contamination reduction in brain mapping.

fNIRS-based online deception decoding

Deception involves complex neural processes in the brain. Different techniques have been used to study and understand brain mechanisms during deception. Moreover, efforts have been made to develop schemes that can detect and differentiate deception and truth-telling. In this paper, a functional near-infrared spectroscopy (fNIRS)-based online brain deception decoding framework is developed. Deploying dual-wavelength fNIRS, we interrogate 16 locations in the forehead when eight able-bodied adults perform deception and truth-telling scenarios separately. By combining preprocessed oxy-hemoglobin and deoxy-hemoglobin signals, we develop subject-specific classifiers using the support vector machine. Deception and truth-telling states are classified correctly in seven out of eight subjects. A control experiment is also conducted to verify the deception-related hemodynamic response. The average classification accuracy is over 83.44% from these seven subjects. The obtained result suggests that the applicability of fNIRS as a brain imaging technique for online deception detection is very promising.