fNIRS-based brain-computer interfaces: a review

Feasibility of Equivalent Dipole Models for Electroencephalogram-Based Brain Computer Interfaces

This article examines the localization errors of equivalent dipolar sources inverted from the surface electroencephalogram in order to determine the feasibility of using their location as classification parameters for non-invasive brain computer interfaces. Inverse localization errors are examined for two head models: a model represented by four concentric spheres and a realistic model based on medical imagery. It is shown that the spherical model results in localization ambiguity such that a number of dipolar sources, with different azimuths and varying orientations, provide a near match to the electroencephalogram of the best equivalent source. No such ambiguity exists for the elevation of inverted sources, indicating that for spherical head models, only the elevation of inverted sources (and not the azimuth) can be expected to provide meaningful classification parameters for brain-computer interfaces. In a realistic head model, all three parameters of the inverted source location are found to be reliable, providing a more robust set of parameters. In both cases, the residual error hypersurfaces demonstrate local minima, indicating that a search for the best-matching sources should be global. Source localization error vs. signal-to-noise ratio is also demonstrated for both head models.

Enhancing Performance of a Hybrid EEG-fNIRS System Using Channel Selection and Early Temporal Features

Brain-Computer Interface (BCI) techniques hold a great promise for neuroprosthetic applications. A desirable BCI system should be portable, minimally invasive, and feature high classification accuracy and efficiency. As two commonly used non-invasive brain imaging modalities, Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) BCI system have often been incorporated in the development of hybrid BCI systems, largely due to their complimentary properties. In this study, we aimed to investigate whether the early temporal information extracted from singular EEG and fNIRS channels on each hemisphere can be used to enhance the accuracy and efficiency of a hybrid EEG-fNIRS BCI system. Eleven healthy volunteers were recruited and underwent simultaneous EEG-fNIRS recording during a motor execution task that included left and right hand movements. Singular EEG and fNIRS channels corresponding to the motor cortices of each hemisphere were selected using a general linear model. Early temporal information was extracted from the EEG channel (0–1 s) along with initial hemodynamic dip information from fNIRS (0–2 s) for classification using a support vector machine (SVM). Results demonstrated a lofty classification accuracy using a minimal number of channels and features derived from early temporal information. In conclusion, a hybrid EEG-fNIRS BCI system can achieve higher classification accuracy (91.02 ± 4.08%) and efficiency by integrating their complimentary properties, compared to using EEG (85.64 ± 7.4%) or fNIRS alone (85.55 ± 10.72%). Such a hybrid system can also achieve minimal response lag in application by focusing on rapidly-evolving brain dynamics.

The Role of Visual Noise in Influencing Mental Load and Fatigue in a Steady-State Motion Visual Evoked Potential-Based Brain-Computer Interface

As a spatial selective attention-based brain-computer interface (BCI) paradigm, steady-state visual evoked potential (SSVEP) BCI has the advantages of high information transfer rate, high tolerance to artifacts, and robust performance across users. However, its benefits come at the cost of mental load and fatigue occurring in the concentration on the visual stimuli. Noise, as a ubiquitous random perturbation with the power of randomness, may be exploited by the human visual system to enhance higher-level brain functions. In this study, a novel steady-state motion visual evoked potential (SSMVEP, i.e., one kind of SSVEP)-based BCI paradigm with spatiotemporal visual noise was used to investigate the influence of noise on the compensation of mental load and fatigue deterioration during prolonged attention tasks. Changes in α, θ, θ + α powers, θ/α ratio, and electroencephalography (EEG) properties of amplitude, signal-to-noise ratio (SNR), and online accuracy, were used to evaluate mental load and fatigue. We showed that presenting a moderate visual noise to participants could reliably alleviate the mental load and fatigue during online operation of visual BCI that places demands on the attentional processes. This demonstrated that noise could provide a superior solution to the implementation of visual attention controlling-based BCI applications.

Assessment and Communication for People with Disorders of Consciousness

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the level of consciousness of people unable to provide motor response and, in a second stage, establishing a communication channel for these people that enables them to answer questions with either 'yes' or 'no'. The suite of paradigms is designed to test basic responses in the first step and to continue to more comprehensive tasks if the first tests are successful. The latter tasks require more cognitive functions, but they could provide communication, which is not possible with the basic tests. All assessment tests produce accuracy plots that show whether the algorithms were able to detect the patient's brain's response to the given tasks. If the accuracy level is beyond the significance level, we assume that the subject understood the task and was able to follow the sequence of commands presented via earphones to the subject. The tasks require users to concentrate on certain stimuli or to imagine moving either the left or right hand. All tasks are designed around the assumption that the user is unable to use the visual modality, and thus, all stimuli presented to the user (including instructions, cues, and feedback) are auditory or tactile.

Multisubject "Learning" for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures

An accurate measure of mental workload level has diverse neuroergonomic applications ranging from brain computer interfacing to improving the efficiency of human operators. In this study, we integrated electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), and physiological measures for the classification of three workload levels in an n-back working memory task. A significantly better than chance level classification was achieved by EEG-alone, fNIRS-alone, physiological alone, and EEG+fNIRS based approaches. The results confirmed our previous finding that integrating EEG and fNIRS significantly improved workload classification compared to using EEG-alone or fNIRS-alone. The inclusion of physiological measures, however, does not significantly improves EEG-based or fNIRS-based workload classification. A major limitation of currently available mental workload assessment approaches is the requirement to record lengthy calibration data from the target subject to train workload classifiers. We show that by learning from the data of other subjects, workload classification accuracy can be improved especially when the amount of data from the target subject is small.

Hybrid Brain-Computer Interface Techniques for Improved Classification Accuracy and Increased Number of Commands: A Review

In this article, non-invasive hybrid brain-computer interface (hBCI) technologies for improving classification accuracy and increasing the number of commands are reviewed. Hybridization combining more than two modalities is a new trend in brain imaging and prosthesis control. Electroencephalography (EEG), due to its easy use and fast temporal resolution, is most widely utilized in combination with other brain/non-brain signal acquisition modalities, for instance, functional near infrared spectroscopy (fNIRS), electromyography (EMG), electrooculography (EOG), and eye tracker. Three main purposes of hybridization are to increase the number of control commands, improve classification accuracy and reduce the signal detection time. Currently, such combinations of EEG + fNIRS and EEG + EOG are most commonly employed. Four principal components (i.e., hardware, paradigm, classifiers, and features) relevant to accuracy improvement are discussed. In the case of brain signals, motor imagination/movement tasks are combined with cognitive tasks to increase active brain-computer interface (BCI) accuracy. Active and reactive tasks sometimes are combined: motor imagination with steady-state evoked visual potentials (SSVEP) and motor imagination with P300. In the case of reactive tasks, SSVEP is most widely combined with P300 to increase the number of commands. Passive BCIs, however, are rare. After discussing the hardware and strategies involved in the development of hBCI, the second part examines the approaches used to increase the number of control commands and to enhance classification accuracy. The future prospects and the extension of hBCI in real-time applications for daily life scenarios are provided.

Virtual and Actual Humanoid Robot Control with Four-Class Motor-Imagery-Based Optical Brain-Computer Interface

Motor-imagery tasks are a popular input method for controlling brain-computer interfaces (BCIs), partially due to their similarities to naturally produced motor signals. The use of functional near-infrared spectroscopy (fNIRS) in BCIs is still emerging and has shown potential as a supplement or replacement for electroencephalography. However, studies often use only two or three motor-imagery tasks, limiting the number of available commands. In this work, we present the results of the first four-class motor-imagery-based online fNIRS-BCI for robot control. Thirteen participants utilized upper- and lower-limb motor-imagery tasks (left hand, right hand, left foot, and right foot) that were mapped to four high-level commands (turn left, turn right, move forward, and move backward) to control the navigation of a simulated or real robot. A significant improvement in classification accuracy was found between the virtual-robot-based BCI (control of a virtual robot) and the physical-robot BCI (control of the DARwIn-OP humanoid robot). Differences were also found in the oxygenated hemoglobin activation patterns of the four tasks between the first and second BCI. These results corroborate previous findings that motor imagery can be improved with feedback and imply that a four-class motor-imagery-based fNIRS-BCI could be feasible with sufficient subject training.

Enhancing Classification Performance of Functional Near-Infrared Spectroscopy- Brain-Computer Interface Using Adaptive Estimation of General Linear Model Coefficients

In this paper, a novel methodology for enhanced classification of functional near-infrared spectroscopy (fNIRS) signals utilizable in a two-class [motor imagery (MI) and rest; mental rotation (MR) and rest] brain–computer interface (BCI) is presented. First, fNIRS signals corresponding to MI and MR are acquired from the motor and prefrontal cortex, respectively, afterward, filtered to remove physiological noises. Then, the signals are modeled using the general linear model, the coefficients of which are adaptively estimated using the least squares technique. Subsequently, multiple feature combinations of estimated coefficients were used for classification. The best classification accuracies achieved for five subjects, for MI versus rest are 79.5, 83.7, 82.6, 81.4, and 84.1% whereas those for MR versus rest are 85.5, 85.2, 87.8, 83.7, and 84.8%, respectively, using support vector machine. These results are compared with the best classification accuracies obtained using the conventional hemodynamic response. By means of the proposed methodology, the average classification accuracy obtained was significantly higher (p < 0.05). These results serve to demonstrate the feasibility of developing a high-classification-performance fNIRS-BCI.

Measuring Mental Workload with EEG+fNIRS

We studied the capability of a Hybrid functional neuroimaging technique to quantify human mental workload (MWL). We have used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) as imaging modalities with 17 healthy subjects performing the letter n-back task, a standard experimental paradigm related to working memory (WM). The level of MWL was parametrically changed by variation of n from 0 to 3. Nineteen EEG channels were covering the whole-head and 19 fNIRS channels were located on the forehead to cover the most dominant brain region involved in WM. Grand block averaging of recorded signals revealed specific behaviors of oxygenated-hemoglobin level during changes in the level of MWL. A machine learning approach has been utilized for detection of the level of MWL. We extracted different features from EEG, fNIRS, and EEG+fNIRS signals as the biomarkers of MWL and fed them to a linear support vector machine (SVM) as train and test sets. These features were selected based on their sensitivity to the changes in the level of MWL according to the literature. We introduced a new category of features within fNIRS and EEG+fNIRS systems. In addition, the performance level of each feature category was systematically assessed. We also assessed the effect of number of features and window size in classification performance. SVM classifier used in order to discriminate between different combinations of cognitive states from binary- and multi-class states. In addition to the cross-validated performance level of the classifier other metrics such as sensitivity, specificity, and predictive values were calculated for a comprehensive assessment of the classification system. The Hybrid (EEG+fNIRS) system had an accuracy that was significantly higher than that of either EEG or fNIRS. Our results suggest that EEG+fNIRS features combined with a classifier are capable of robustly discriminating among various levels of MWL. Results suggest that EEG+fNIRS should be preferred to only EEG or fNIRS, in developing passive BCIs and other applications which need to monitor users' MWL.

Motor Imagery EEG Classification for Patients with Amyotrophic Lateral Sclerosis Using Fractal Dimension and Fisher's Criterion-Based Channel Selection

Motor imagery is based on the volitional modulation of sensorimotor rhythms (SMRs); however, the sensorimotor processes in patients with amyotrophic lateral sclerosis (ALS) are impaired, leading to degenerated motor imagery ability. Thus, motor imagery classification in ALS patients has been considered challenging in the brain–computer interface (BCI) community. In this study, we address this critical issue by introducing the Grassberger–Procaccia and Higuchi’s methods to estimate the fractal dimensions (GPFD and HFD, respectively) of the electroencephalography (EEG) signals from ALS patients. Moreover, a Fisher’s criterion-based channel selection strategy is proposed to automatically determine the best patient-dependent channel configuration from 30 EEG recording sites. An EEG data collection paradigm is designed to collect the EEG signal of resting state and the imagination of three movements, including right hand grasping (RH), left hand grasping (LH), and left foot stepping (LF). Five late-stage ALS patients without receiving any SMR training participated in this study. Experimental results show that the proposed GPFD feature is not only superior to the previously-used SMR features (mu and beta band powers of EEG from sensorimotor cortex) but also better than HFD. The accuracies achieved by the SMR features are not satisfactory (all lower than 80%) in all binary classification tasks, including RH imagery vs. resting, LH imagery vs. resting, and LF imagery vs. resting. For the discrimination between RH imagery and resting, the average accuracies of GPFD in 30-channel (without channel selection) and top-five-channel configurations are 95.25% and 93.50%, respectively. When using only one channel (the best channel among the 30), a high accuracy of 91.00% can still be achieved by the GPFD feature and a linear discriminant analysis (LDA) classifier. The results also demonstrate that the proposed Fisher’s criterion-based channel selection is capable of removing a large amount of redundant and noisy EEG channels. The proposed GPFD feature extraction combined with the channel selection strategy can be used as the basis for further developing high-accuracy and high-usability motor imagery BCI systems from which the patients with ALS can really benefit.

Relevant Feature Integration and Extraction for Single-Trial Motor Imagery Classification

Brain computer interfaces provide a novel channel for the communication between brain and output devices. The effectiveness of the brain computer interface is based on the classification accuracy of single trial brain signals. The common spatial pattern (CSP) algorithm is believed to be an effective algorithm for the classification of single trial brain signals. As the amplitude feature for spatial projection applied by this algorithm is based on a broad frequency bandpass filter (mainly 5–30 Hz) in which the frequency band is often selected by experience, the CSP is sensitive to noise and the influence of other irrelevant information in the selected broad frequency band. In this paper, to improve the CSP, a novel relevant feature integration and extraction algorithm is proposed. Before projecting, we integrated the motor relevant information to suppress the interference of noise and irrelevant information, as well as to improve the spatial difference for projection. The algorithm was evaluated with public datasets. It showed significantly better classification performance with single trial electroencephalography (EEG) data, increasing by 6.8% compared with the CSP.

A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition

A Brain-Computer Interface (BCI) is a setup permitting the control of external devices by decoding brain activity. Electroencephalography (EEG) has been extensively used for decoding brain activity since it is non-invasive, cheap, portable, and has high temporal resolution to allow real-time operation. Due to its poor spatial specificity, BCIs based on EEG can require extensive training and multiple trials to decode brain activity (consequently slowing down the operation of the BCI). On the other hand, BCIs based on functional magnetic resonance imaging (fMRI) are more accurate owing to its superior spatial resolution and sensitivity to underlying neuronal processes which are functionally localized. However, due to its relatively low temporal resolution, high cost, and lack of portability, fMRI is unlikely to be used for routine BCI. We propose a new approach for transferring the capabilities of fMRI to EEG, which includes simultaneous EEG/fMRI sessions for finding a mapping from EEG to fMRI, followed by a BCI run from only EEG data, but driven by fMRI-like features obtained from the mapping identified previously. Our novel data-driven method is likely to discover latent linkages between electrical and hemodynamic signatures of neural activity hitherto unexplored using model-driven methods, and is likely to serve as a template for a novel multi-modal strategy wherein cross-modal EEG-fMRI interactions are exploited for the operation of a unimodal EEG system, leading to a new generation of EEG-based BCIs.

A Gaussian mixture model based adaptive classifier for fNIRS brain-computer interfaces and its testing via simulation

OBJECTIVE

Functional near infra-red spectroscopy (fNIRS) is a promising brain imaging technology for brain-computer interfaces (BCI). Future clinical uses of fNIRS will likely require operation over long time spans, during which neural activation patterns may change. However, current decoders for fNIRS signals are not designed to handle changing activation patterns. The objective of this study is to test via simulations a new adaptive decoder for fNIRS signals, the Gaussian mixture model adaptive classifier (GMMAC).

APPROACH

GMMAC can simultaneously classify and track activation pattern changes without the need for ground-truth labels. This adaptive classifier uses computationally efficient variational Bayesian inference to label new data points and update mixture model parameters, using the previous model parameters as priors. We test GMMAC in simulations in which neural activation patterns change over time and compare to static decoders and unsupervised adaptive linear discriminant analysis classifiers.

MAIN RESULTS

Our simulation experiments show GMMAC can accurately decode under time-varying activation patterns: shifts of activation region, expansions of activation region, and combined contractions and shifts of activation region. Furthermore, the experiments show the proposed method can track the changing shape of the activation region. Compared to prior work, GMMAC performed significantly better than the other unsupervised adaptive classifiers on a difficult activation pattern change simulation: 99% versus  <54% in two-choice classification accuracy.

SIGNIFICANCE

We believe GMMAC will be useful for clinical fNIRS-based brain-computer interfaces, including neurofeedback training systems, where operation over long time spans is required.

Object Recognition in Mental Representations: Directions for Exploring Diagnostic Features through Visual Mental Imagery

One of the fundamental goals of object recognition research is to understand how a cognitive representation produced from the output of filtered and transformed sensory information facilitates efficient viewer behavior. Given that mental imagery strongly resembles perceptual processes in both cortical regions and subjective visual qualities, it is reasonable to question whether mental imagery facilitates cognition in a manner similar to that of perceptual viewing: via the detection and recognition of distinguishing features. Categorizing the feature content of mental imagery holds potential as a reverse pathway by which to identify the components of a visual stimulus which are most critical for the creation and retrieval of a visual representation. This review will examine the likelihood that the information represented in visual mental imagery reflects distinctive object features thought to facilitate efficient object categorization and recognition during perceptual viewing. If it is the case that these representational features resemble their sensory counterparts in both spatial and semantic qualities, they may well be accessible through mental imagery as evaluated through current investigative techniques. In this review, methods applied to mental imagery research and their findings are reviewed and evaluated for their efficiency in accessing internal representations, and implications for identifying diagnostic features are discussed. An argument is made for the benefits of combining mental imagery assessment methods with diagnostic feature research to advance the understanding of visual perceptive processes, with suggestions for avenues of future investigation.

Comparison of Brain Activation during Motor Imagery and Motor Movement Using fNIRS

Motor-activity-related mental tasks are widely adopted for brain-computer interfaces (BCIs) as they are a natural extension of movement intention, requiring no training to evoke brain activity. The ideal BCI aims to eliminate neuromuscular movement, making motor imagery tasks, or imagined actions with no muscle movement, good candidates. This study explores cortical activation differences between motor imagery and motor execution for both upper and lower limbs using functional near-infrared spectroscopy (fNIRS). Four simple finger- or toe-tapping tasks (left hand, right hand, left foot, and right foot) were performed with both motor imagery and motor execution and compared to resting state. Significant activation was found during all four motor imagery tasks, indicating that they can be detected via fNIRS. Motor execution produced higher activation levels, a faster response, and a different spatial distribution compared to motor imagery, which should be taken into account when designing an imagery-based BCI. When comparing left versus right, upper limb tasks are the most clearly distinguishable, particularly during motor execution. Left and right lower limb activation patterns were found to be highly similar during both imagery and execution, indicating that higher resolution imaging, advanced signal processing, or improved subject training may be required to reliably distinguish them.

A novel GLM-based method for the Automatic IDentification of functional Events (AIDE) in fNIRS data recorded in naturalistic environments

Recent technological advances have allowed the development of portable functional Near-Infrared Spectroscopy (fNIRS) devices that can be used to perform neuroimaging in the real-world. However, as real-world experiments are designed to mimic everyday life situations, the identification of event onsets can be extremely challenging and time-consuming. Here, we present a novel analysis method based on the general linear model (GLM) least square fit analysis for the Automatic IDentification of functional Events (or AIDE) directly from real-world fNIRS neuroimaging data. In order to investigate the accuracy and feasibility of this method, as a proof-of-principle we applied the algorithm to (i) synthetic fNIRS data simulating both block-, event-related and mixed-design experiments and (ii) experimental fNIRS data recorded during a conventional lab-based task (involving maths). AIDE was able to recover functional events from simulated fNIRS data with an accuracy of 89%, 97% and 91% for the simulated block-, event-related and mixed-design experiments respectively. For the lab-based experiment, AIDE recovered more than the 66.7% of the functional events from the fNIRS experimental measured data. To illustrate the strength of this method, we then applied AIDE to fNIRS data recorded by a wearable system on one participant during a complex real-world prospective memory experiment conducted outside the lab. As part of the experiment, there were four and six events (actions where participants had to interact with a target) for the two different conditions respectively (condition 1: social-interact with a person; condition 2: non-social-interact with an object). AIDE managed to recover 3/4 events and 3/6 events for conditions 1 and 2 respectively. The identified functional events were then corresponded to behavioural data from the video recordings of the movements and actions of the participant. Our results suggest that "brain-first" rather than "behaviour-first" analysis is possible and that the present method can provide a novel solution to analyse real-world fNIRS data, filling the gap between real-life testing and functional neuroimaging.

Hand Motion Detection in fNIRS Neuroimaging Data

As the number of people diagnosed with movement disorders is increasing, it becomes vital to design techniques that allow the better understanding of human brain in naturalistic settings. There are many brain imaging methods such as fMRI, SPECT, and MEG that provide the functional information of the brain. However, these techniques have some limitations including immobility, cost, and motion artifacts. One of the most emerging portable brain scanners available today is functional near-infrared spectroscopy (fNIRS). In this study, we have conducted fNIRS neuroimaging of seven healthy subjects while they were performing wrist tasks such as flipping their hand with the periods of rest (no movement). Different models of support vector machine is applied to these fNIRS neuroimaging data and the results show that we could classify the action and rest periods with the accuracy of over 80% for the fNIRS data of individual participants. Our results are promising and suggest that the presented classification method for fNIRS could further be applied to real-time applications such as brain computer interfacing (BCI), and into the future steps of this research to record brain activity from fNIRS and EEG, and fuse them with the body motion sensors to correlate the activities.

Lawrence K. Can time-resolved NIRS provide the sensitivity to detect brain activity during motor imagery consistently?

Previous functional magnetic resonance imaging (fMRI) studies have shown that a subgroup of patients diagnosed as being in a vegetative state are aware and able to communicate by performing a motor imagery task in response to commands. Due to the fMRI's cost and accessibility, there is a need for exploring different imaging modalities that can be used at the bedside. A promising technique is functional near infrared spectroscopy (fNIRS) that has been successfully applied to measure brain oxygenation in humans. Due to the limited depth sensitivity of continuous-wave NIRS, time-resolved (TR) detection has been proposed as a way of enhancing the sensitivity to the brain, since late arriving photons have a higher probability of reaching the brain. The goal of this study was to assess the feasibility and sensitivity of TR fNIRS in detecting brain activity during motor imagery. Fifteen healthy subjects were recruited in this study, and the fNIRS results were validated using fMRI. The change in the statistical moments of the distribution of times of flight (number of photons, mean time of flight and variance) were calculated for each channel to determine the presence of brain activity. The results indicate up to an 86% agreement between fMRI and TR-fNIRS and the sensitivity ranging from 64 to 93% with the highest value determined for the mean time of flight. These promising results highlight the potential of TR-fNIRS as a portable brain computer interface for patients with disorder of consciousness.

Hybrid EEG-fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control

In this paper, a hybrid electroencephalography–functional near-infrared spectroscopy (EEG–fNIRS) scheme to decode eight active brain commands from the frontal brain region for brain–computer interface is presented. A total of eight commands are decoded by fNIRS, as positioned on the prefrontal cortex, and by EEG, around the frontal, parietal, and visual cortices. Mental arithmetic, mental counting, mental rotation, and word formation tasks are decoded with fNIRS, in which the selected features for classification and command generation are the peak, minimum, and mean ΔHbO values within a 2-s moving window. In the case of EEG, two eyeblinks, three eyeblinks, and eye movement in the up/down and left/right directions are used for four-command generation. The features in this case are the number of peaks and the mean of the EEG signal during 1 s window. We tested the generated commands on a quadcopter in an open space. An average accuracy of 75.6% was achieved with fNIRS for four-command decoding and 86% with EEG for another four-command decoding. The testing results show the possibility of controlling a quadcopter online and in real-time using eight commands from the prefrontal and frontal cortices via the proposed hybrid EEG–fNIRS interface.

A Non-parametric Approach to the Overall Estimate of Cognitive Load Using NIRS Time Series

We present a non-parametric approach to prediction of the n-back n ∈ {1, 2} task as a proxy measure of mental workload using Near Infrared Spectroscopy (NIRS) data. In particular, we focus on measuring the mental workload through hemodynamic responses in the brain induced by these tasks, thereby realizing the potential that they can offer for their detection in real world scenarios (e.g., difficulty of a conversation). Our approach takes advantage of intrinsic linearity that is inherent in the components of the NIRS time series to adopt a one-step regression strategy. We demonstrate the correctness of our approach through its mathematical analysis. Furthermore, we study the performance of our model in an inter-subject setting in contrast with state-of-the-art techniques in the literature to show a significant improvement on prediction of these tasks (82.50 and 86.40% for female and male participants, respectively). Moreover, our empirical analysis suggest a gender difference effect on the performance of the classifiers (with male data exhibiting a higher non-linearity) along with the left-lateralized activation in both genders with higher specificity in females.

Competing Biases in Mental Arithmetic: When Division Is More and Multiplication Is Less

Mental arithmetic exhibits various biases. Among those is a tendency to overestimate addition and to underestimate subtraction outcomes. Does such "operational momentum" (OM) also affect multiplication and division? Twenty-six adults produced lines whose lengths corresponded to the correct outcomes of multiplication and division problems shown in symbolic format. We found a reliable tendency to over-estimate division outcomes, i.e., reverse OM. We suggest that anchoring on the first operand (a tendency to use this number as a reference for further quantitative reasoning) contributes to cognitive biases in mental arithmetic.

BCI Control of Heuristic Search Algorithms

The ability to develop Brain-Computer Interfaces (BCI) to Intelligent Systems would offer new perspectives in terms of human supervision of complex Artificial Intelligence (AI) systems, as well as supporting new types of applications. In this article, we introduce a basic mechanism for the control of heuristic search through fNIRS-based BCI. The rationale is that heuristic search is not only a basic AI mechanism but also one still at the heart of many different AI systems. We investigate how users’ mental disposition can be harnessed to influence the performance of heuristic search algorithm through a mechanism of precision-complexity exchange. From a system perspective, we use weighted variants of the A* algorithm which have an ability to provide faster, albeit suboptimal solutions. We use recent results in affective BCI to capture a BCI signal, which is indicative of a compatible mental disposition in the user. It has been established that Prefrontal Cortex (PFC) asymmetry is strongly correlated to motivational dispositions and results anticipation, such as approach or even risk-taking, and that this asymmetry is amenable to Neurofeedback (NF) control. Since PFC asymmetry is accessible through fNIRS, we designed a BCI paradigm in which users vary their PFC asymmetry through NF during heuristic search tasks, resulting in faster solutions. This is achieved through mapping the PFC asymmetry value onto the dynamic weighting parameter of the weighted A* (WA*) algorithm. We illustrate this approach through two different experiments, one based on solving 8-puzzle configurations, and the other on path planning. In both experiments, subjects were able to speed up the computation of a solution through a reduction of search space in WA*. Our results establish the ability of subjects to intervene in heuristic search progression, with effects which are commensurate to their control of PFC asymmetry: this opens the way to new mechanisms for the implementation of hybrid cognitive systems.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

Detection and classification of three-class initial dips from prefrontal cortex

In this paper, the use of initial dips using functional near-infrared spectroscopy (fNIRS) for brain-computer interface (BCI) is investigated. Features and window sizes for detecting initial dips are also discussed. Three mental tasks including mental arithmetic, mental counting, and puzzle solving are performed in obtaining fNIRS signals from the prefrontal cortex. Vector-based phase analysis method combined with a threshold circle, as a decision criterion, are used to detect the initial dips. Eight healthy subjects participate in experiment. Linear discriminant analysis is used as a classifier. To classify initial dips, five features (signal mean, peak value, signal slope, skewness, and kurtosis) of oxy-hemoglobin (HbO) and four different window sizes (0~1, 0~1.5, 0~2, and 0~2.5 sec) are examined. It is shown that a combination of signal mean and peak value and a time period of 0~2.5 sec provide the best average classification accuracy of 57.5% for three classes. To further validate the result, three-class classification using the conventional hemodynamic response (HR) is also performed, in which two features (signal mean and signal slope) and 2~7 sec window size have yielded the average classification accuracy of 65.9%. This reveals that fNIRS-based BCI using initial dip detection can reduce the command generation time from 7 sec to 2.5 sec while the classification accuracy is a bit sacrificed from 65.9% to 57.5% for three mental tasks. Further improvement can be made by using deoxy hemoglobin signals in coping with the slow HR problem.

Left Lateral Prefrontal Activity Reflects a Change of Behavioral Tactics to Cope with a Given Rule: An fNIRS Study

Rules prescribe human behavior and our attempts to choose appropriate behavior under a given rule. Cognitive control, a mechanism to choose and evaluate actions under a rule, is required to determine the appropriate behavior within the limitations of that rule. Consequently, such cognitive control increases mental workload. However, the workload caused by a cognitive task might be different when an additional rule must be considered in choosing the action. The present study was a functional near-infrared spectroscopy (fNIRS) investigation of an experimental task, in which the difficulty of an operation and existence of an additional rule were manipulated to dissociate the influence of that additional rule on cognitive processing. Twenty healthy Japanese volunteers participated. The participants performed an experimental task, in which the player caught one of five colored balls from the upper part of a computer screen by operating a mouse. Four task conditions were prepared to manipulate the task difficulty, which was defined in terms of operational difficulty. In turn, operational difficulty was determined by the width of the playable space and the existence of an additional rule, which reduced the score when a red ball was not caught. The 52-channel fNIRS data were collected from the forehead. Two regions of interest (ROIs) associated with the bilateral lateral prefrontal cortices (LPFCs) were determined, and a three-way repeated-measures analysis of variance (ANOVA) was performed using the task-related signal changes from each ROI. The fNIRS results revealed that bilateral LPFCs showed large signal changes with the increase in mental workload. The ANOVA showed a significant interaction between the existence of an additional rule and the location of the ROIs; that is, the left lateral prefrontal area showed a significant increase in signal intensity when the additional rule existed, and the participant occasionally decided to avoid catching a ball to successfully catch the red-colored ball. Thus, activation of the left LPFC corresponded more closely to the increase in cognitive control underlying the behavioral change made to cope with the additional rule.

Decomposition of Near-Infrared Spectroscopy Signals Using Oblique Subspace Projections: Applications in Brain Hemodynamic Monitoring

Clinical data is comprised by a large number of synchronously collected biomedical signals that are measured at different locations. Deciphering the interrelationships of these signals can yield important information about their dependence providing some useful clinical diagnostic data. For instance, by computing the coupling between Near-Infrared Spectroscopy signals (NIRS) and systemic variables the status of the hemodynamic regulation mechanisms can be assessed. In this paper we introduce an algorithm for the decomposition of NIRS signals into additive components. The algorithm, SIgnal DEcomposition base on Obliques Subspace Projections (SIDE-ObSP), assumes that the measured NIRS signal is a linear combination of the systemic measurements, following the linear regression model y = Ax + ϵ. SIDE-ObSP decomposes the output such that, each component in the decomposition represents the sole linear influence of one corresponding regressor variable. This decomposition scheme aims at providing a better understanding of the relation between NIRS and systemic variables, and to provide a framework for the clinical interpretation of regression algorithms, thereby, facilitating their introduction into clinical practice. SIDE-ObSP combines oblique subspace projections (ObSP) with the structure of a mean average system in order to define adequate signal subspaces. To guarantee smoothness in the estimated regression parameters, as observed in normal physiological processes, we impose a Tikhonov regularization using a matrix differential operator. We evaluate the performance of SIDE-ObSP by using a synthetic dataset, and present two case studies in the field of cerebral hemodynamics monitoring using NIRS. In addition, we compare the performance of this method with other system identification techniques. In the first case study data from 20 neonates during the first 3 days of life was used, here SIDE-ObSP decoupled the influence of changes in arterial oxygen saturation from the NIRS measurements, facilitating the use of NIRS as a surrogate measure for cerebral blood flow (CBF). The second case study used data from a 3-years old infant under Extra Corporeal Membrane Oxygenation (ECMO), here SIDE-ObSP decomposed cerebral/peripheral tissue oxygenation, as a sum of the partial contributions from different systemic variables, facilitating the comparison between the effects of each systemic variable on the cerebral/peripheral hemodynamics.

Bundled-Optode Method in Functional Near-Infrared Spectroscopy

In this paper, a theory for detection of the absolute concentrations of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) from hemodynamic responses using a bundled-optode configuration in functional near-infrared spectroscopy (fNIRS) is proposed. The proposed method is then applied to the identification of two fingers (i.e., little and thumb) during their flexion and extension. This experiment involves a continuous-wave-type dual-wavelength (760 and 830 nm) fNIRS and five healthy male subjects. The active brain locations of two finger movements are identified based on the analysis of the t- and p-values of the averaged HbOs, which are quite distinctive. Our experimental results, furthermore, revealed that the hemodynamic responses of two-finger movements are different: The mean, peak, and time-to-peak of little finger movements are higher than those of thumb movements. It is noteworthy that the developed method can be extended to 3-dimensional fNIRS imaging.

Analysis of Different Classification Techniques for Two-Class Functional Near-Infrared Spectroscopy-Based Brain-Computer Interface

We analyse and compare the classification accuracies of six different classifiers for a two-class mental task (mental arithmetic and rest) using functional near-infrared spectroscopy (fNIRS) signals. The signals of the mental arithmetic and rest tasks from the prefrontal cortex region of the brain for seven healthy subjects were acquired using a multichannel continuous-wave imaging system. After removal of the physiological noises, six features were extracted from the oxygenated hemoglobin (HbO) signals. Two- and three-dimensional combinations of those features were used for classification of mental tasks. In the classification, six different modalities, linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), k-nearest neighbour (kNN), the Naïve Bayes approach, support vector machine (SVM), and artificial neural networks (ANN), were utilized. With these classifiers, the average classification accuracies among the seven subjects for the 2- and 3-dimensional combinations of features were 71.6, 90.0, 69.7, 89.8, 89.5, and 91.4% and 79.6, 95.2, 64.5, 94.8, 95.2, and 96.3%, respectively. ANN showed the maximum classification accuracies: 91.4 and 96.3%. In order to validate the results, a statistical significance test was performed, which confirmed that the p values were statistically significant relative to all of the other classifiers (p < 0.005) using HbO signals.

Toward more intuitive brain-computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy

In traditional brain-computer interface (BCI) studies, binary communication systems have generally been implemented using two mental tasks arbitrarily assigned to “yes” or “no” intentions (e.g., mental arithmetic calculation for “yes”). A recent pilot study performed with one paralyzed patient showed the possibility of a more intuitive paradigm for binary BCI communications, in which the patient’s internal yes/no intentions were directly decoded from functional near-infrared spectroscopy (fNIRS). We investigated whether such an “fNIRS-based direct intention decoding” paradigm can be reliably used for practical BCI communications. Eight healthy subjects participated in this study, and each participant was administered 70 disjunctive questions. Brain hemodynamic responses were recorded using a multichannel fNIRS device, while the participants were internally expressing “yes” or “no” intentions to each question. Different feature types, feature numbers, and time window sizes were tested to investigate optimal conditions for classifying the internal binary intentions. About 75% of the answers were correctly classified when the individual best feature set was employed (75.89% ± 1.39 and 74.08% ± 2.87 for oxygenated and deoxygenated hemoglobin responses, respectively), which was significantly higher than a random chance level (68.57% for p < 0.001). The kurtosis feature showed the highest mean classification accuracy among all feature types. The grand-averaged hemodynamic responses showed that wide brain regions are associated with the processing of binary implicit intentions. Our experimental results demonstrated that direct decoding of internal binary intention has the potential to be used for implementing more intuitive and user-friendly communication systems for patients with motor disabilities.

Review of functional near-infrared spectroscopy in neurorehabilitation

We provide a brief overview of the research and clinical applications of near-infrared spectroscopy (NIRS) in the neurorehabilitation field. NIRS has several potential advantages and shortcomings as a neuroimaging tool and is suitable for research application in the rehabilitation field. As one of the main applications of NIRS, we discuss its application as a monitoring tool, including investigating the neural mechanism of functional recovery after brain damage and investigating the neural mechanisms for controlling bipedal locomotion and postural balance in humans. In addition to being a monitoring tool, advances in signal processing techniques allow us to use NIRS as a therapeutic tool in this field. With a brief summary of recent studies investigating the clinical application of NIRS using motor imagery task, we discuss the possible clinical usage of NIRS in brain-computer interface and neurofeedback.

Prefrontal Cortex Activation While Walking Under Dual-Task Conditions in Stroke: A Multimodal Imaging Study

Background Walking while performing another task (eg, talking) is challenging for many stroke survivors, yet its neural basis are not fully understood. Objective To investigate prefrontal cortex activation and its relationship to gait measures while walking under single-task (ST) and dual-task (DT) conditions (ie, walking while simultaneously performing a cognitive task) in stroke survivors. Methods We acquired near-infrared spectroscopy (NIRS) data from the prefrontal cortex during treadmill walking in ST and DT conditions in chronic stroke survivors and healthy controls. We also acquired functional magnetic resonance imaging (fMRI) and NIRS during simulated walking under these conditions. Results NIRS revealed increased oxygenated hemoglobin concentration in DT-walking compared with ST-walking for both groups. For simulated walking, NIRS showed a significant effect of group and group × task, being greater on both occasions, in stroke survivors. A greater increase in brain activation observed from ST to DT walking/ simulated walking was related to a greater change in motor performance in stroke survivors. fMRI revealed increased activity during DT relative to ST conditions in stroke patients in areas including the inferior temporal gyri, superior frontal gyri and cingulate gyri bilaterally, and the right precentral gyrus. The DT-related increase in fMRI activity correlated with DT-related change in behavior in stroke participants in the bilateral inferior temporal gyrus, left cingulate gyrus, and left frontal pole. Conclusion Our results provide novel evidence that enhanced brain activity changes relate to dual task motor decrements.

Cortical Signal Analysis and Advances in Functional Near-Infrared Spectroscopy Signal: A Review

Functional near-infrared spectroscopy (fNIRS) is a non-invasive neuroimaging modality that measures the concentration changes of oxy-hemoglobin (HbO) and de-oxy hemoglobin (HbR) at the same time. It is an emerging cortical imaging modality with a good temporal resolution that is acceptable for brain-computer interface applications. Researchers have developed several methods in last two decades to extract the neuronal activation related waveform from the observed fNIRS time series. But still there is no standard method for analysis of fNIRS data. This article presents a brief review of existing methodologies to model and analyze the activation signal. The purpose of this review article is to give a general overview of variety of existing methodologies to extract useful information from measured fNIRS data including pre-processing steps, effects of differential path length factor (DPF), variations and attributes of hemodynamic response function (HRF), extraction of evoked response, removal of physiological noises, instrumentation, and environmental noises and resting/activation state functional connectivity. Finally, the challenges in the analysis of fNIRS signal are summarized.

How feedback, motor imagery, and reward influence brain self-regulation using real-time fMRI

The learning process involved in achieving brain self-regulation is presumed to be related to several factors, such as type of feedback, reward, mental imagery, duration of training, among others. Explicitly instructing participants to use mental imagery and monetary reward are common practices in real-time fMRI (rtfMRI) neurofeedback (NF), under the assumption that they will enhance and accelerate the learning process. However, it is still not clear what the optimal strategy is for improving volitional control. We investigated the differential effect of feedback, explicit instructions and monetary reward while training healthy individuals to up-regulate the blood-oxygen-level dependent (BOLD) signal in the supplementary motor area (SMA). Four groups were trained in a two-day rtfMRI-NF protocol: GF with NF only, GF,I with NF + explicit instructions (motor imagery), GF,R with NF + monetary reward, and GF,I,R with NF + explicit instructions (motor imagery) + monetary reward. Our results showed that GF increased significantly their BOLD self-regulation from day-1 to day-2 and GF,R showed the highest BOLD signal amplitude in SMA during the training. The two groups who were instructed to use motor imagery did not show a significant learning effect over the 2 days. The additional factors, namely motor imagery and reward, tended to increase the intersubject variability in the SMA during the course of training. Whole brain univariate and functional connectivity analyses showed common as well as distinct patterns in the four groups, representing the varied influences of feedback, reward, and instructions on the brain. Hum Brain Mapp 37:3153-3171, 2016. © 2016 Wiley Periodicals, Inc.

Determining Optimal Feature-Combination for LDA Classification of Functional Near-Infrared Spectroscopy Signals in Brain-Computer Interface Application

In this study, we determine the optimal feature-combination for classification of functional near-infrared spectroscopy (fNIRS) signals with the best accuracies for development of a two-class brain-computer interface (BCI). Using a multi-channel continuous-wave imaging system, mental arithmetic signals are acquired from the prefrontal cortex of seven healthy subjects. After removing physiological noises, six oxygenated and deoxygenated hemoglobin (HbO and HbR) features-mean, slope, variance, peak, skewness and kurtosis-are calculated. All possible 2- and 3-feature combinations of the calculated features are then used to classify mental arithmetic vs. rest using linear discriminant analysis (LDA). It is found that the combinations containing mean and peak values yielded significantly higher (p < 0.05) classification accuracies for both HbO and HbR than did all of the other combinations, across all of the subjects. These results demonstrate the feasibility of achieving high classification accuracies using mean and peak values of HbO and HbR as features for classification of mental arithmetic vs. rest for a two-class BCI.

Unsupervised Event Characterization and Detection in Multichannel Signals: An EEG application

In this paper, we propose a new unsupervised method to automatically characterize and detect events in multichannel signals. This method is used to identify artifacts in electroencephalogram (EEG) recordings of brain activity. The proposed algorithm has been evaluated and compared with a supervised method. To this end an example of the performance of the algorithm to detect artifacts is shown. The results show that although both methods obtain similar classification, the proposed method allows detecting events without training data and can also be applied in signals whose events are unknown a priori. Furthermore, the proposed method provides an optimal window whereby an optimal detection and characterization of events is found. The detection of events can be applied in real-time.

Reduction of Delay in Detecting Initial Dips from Functional Near-Infrared Spectroscopy Signals Using Vector-Based Phase Analysis

In this paper, we present a systematic method to reduce the time lag in detecting initial dips using a vector-based phase diagram and an autoregressive moving average with exogenous signals (ARMAX) model-based [Formula: see text]-step-ahead prediction algorithm. With functional near-infrared spectroscopy (fNIRS), signals related to mental arithmetic and right-hand clenching are acquired from the prefrontal and left primary motor cortices, respectively. The interrelationship between oxygenated hemoglobin, deoxygenated hemoglobin, total hemoglobin and cerebral oxygen exchange are related to initial dips. Specifically, a threshold value from the resting state hemodynamics is incorporated, as a decision criterion, into the vector-based phase diagram to determine the occurrence of initial dips. To further reduce the time lag, a [Formula: see text]-step-ahead prediction method is applied to predict the occurrence of the dips. A combination of the threshold criterion and the prediction method resulted in the delay time of about 0.9[Formula: see text]s. The results demonstrate that rapid detection of initial dip is possible and therefore can be used for real-time brain–computer interfacing.

Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Functional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.

A Wearable Channel Selection-Based Brain-Computer Interface for Motor Imagery Detection

Motor imagery-based brain-computer interface (BCI) is a communication interface between an external machine and the brain. Many kinds of spatial filters are used in BCIs to enhance the electroencephalography (EEG) features related to motor imagery. The approach of channel selection, developed to reserve meaningful EEG channels, is also an important technique for the development of BCIs. However, current BCI systems require a conventional EEG machine and EEG electrodes with conductive gel to acquire multi-channel EEG signals and then transmit these EEG signals to the back-end computer to perform the approach of channel selection. This reduces the convenience of use in daily life and increases the limitations of BCI applications. In order to improve the above issues, a novel wearable channel selection-based brain-computer interface is proposed. Here, retractable comb-shaped active dry electrodes are designed to measure the EEG signals on a hairy site, without conductive gel. By the design of analog CAR spatial filters and the firmware of EEG acquisition module, the function of spatial filters could be performed without any calculation, and channel selection could be performed in the front-end device to improve the practicability of detecting motor imagery in the wearable EEG device directly or in commercial mobile phones or tablets, which may have relatively low system specifications. Finally, the performance of the proposed BCI is investigated, and the experimental results show that the proposed system is a good wearable BCI system prototype.

Processing Functional Near Infrared Spectroscopy Signal with a Kalman Filter to Assess Working Memory during Simulated Flight

Working memory (WM) is a key executive function for operating aircraft, especially when pilots have to recall series of air traffic control instructions. There is a need to implement tools to monitor WM as its limitation may jeopardize flight safety. An innovative way to address this issue is to adopt a Neuroergonomics approach that merges knowledge and methods from Human Factors, System Engineering, and Neuroscience. A challenge of great importance for Neuroergonomics is to implement efficient brain imaging techniques to measure the brain at work and to design Brain Computer Interfaces (BCI). We used functional near infrared spectroscopy as it has been already successfully tested to measure WM capacity in complex environment with air traffic controllers (ATC), pilots, or unmanned vehicle operators. However, the extraction of relevant features from the raw signal in ecological environment is still a critical issue due to the complexity of implementing real-time signal processing techniques without a priori knowledge. We proposed to implement the Kalman filtering approach, a signal processing technique that is efficient when the dynamics of the signal can be modeled. We based our approach on the Boynton model of hemodynamic response. We conducted a first experiment with nine participants involving a basic WM task to estimate the noise covariances of the Kalman filter. We then conducted a more ecological experiment in our flight simulator with 18 pilots who interacted with ATC instructions (two levels of difficulty). The data was processed with the same Kalman filter settings implemented in the first experiment. This filter was benchmarked with a classical pass-band IIR filter and a Moving Average Convergence Divergence (MACD) filter. Statistical analysis revealed that the Kalman filter was the most efficient to separate the two levels of load, by increasing the observed effect size in prefrontal areas involved in WM. In addition, the use of a Kalman filter increased the performance of the classification of WM levels based on brain signal. The results suggest that Kalman filter is a suitable approach for real-time improvement of near infrared spectroscopy signal in ecological situations and the development of BCI.

Enhanced Z-LDA for Small Sample Size Training in Brain-Computer Interface Systems

Background. Usually the training set of online brain-computer interface (BCI) experiment is small. For the small training set, it lacks enough information to deeply train the classifier, resulting in the poor classification performance during online testing. Methods. In this paper, on the basis of Z-LDA, we further calculate the classification probability of Z-LDA and then use it to select the reliable samples from the testing set to enlarge the training set, aiming to mine the additional information from testing set to adjust the biased classification boundary obtained from the small training set. The proposed approach is an extension of previous Z-LDA and is named enhanced Z-LDA (EZ-LDA). Results. We evaluated the classification performance of LDA, Z-LDA, and EZ-LDA on simulation and real BCI datasets with different sizes of training samples, and classification results showed EZ-LDA achieved the best classification performance. Conclusions. EZ-LDA is promising to deal with the small sample size training problem usually existing in online BCI system.

Passive BCI based on drowsiness detection: an fNIRS study

We use functional near-infrared spectroscopy (fNIRS) to discriminate the alert and drowsy states for a passive brain-computer interface (BCI). The passive brain signals for the drowsy state are acquired from the prefrontal and dorsolateral prefrontal cortex. The experiment is performed on 13 healthy subjects using a driving simulator, and their brain activity is recorded using a continuous-wave fNIRS system. Linear discriminant analysis (LDA) is employed for training and testing, using the data from the prefrontal, left- and right-dorsolateral prefrontal regions. For classification, eight features are tested: mean oxyhemoglobin, mean deoxyhemoglobin, skewness, kurtosis, signal slope, number of peaks, sum of peaks, and signal peak, in 0~5, 0~10, and 0~15 second time windows, respectively. The results show that the best performance for classification is achieved using mean oxyhemoglobin, the signal peak, and the sum of peaks as features. The average accuracies in the right dorsolateral prefrontal cortex (83.1, 83.4 and 84.9% in the 0~5, 0~10 and 0~15 second time windows, respectively) show that the proposed method has an effective utility for detection of drowsiness for a passive BCI.

Single-trial lie detection using a combined fNIRS-polygraph system

Deception is a human behavior that many people experience in daily life. It involves complex neuronal activities in addition to several physiological changes in the body. A polygraph, which can measure some of the physiological responses from the body, has been widely employed in lie-detection. Many researchers, however, believe that lie detection can become more precise if the neuronal changes that occur in the process of deception can be isolated and measured. In this study, we combine both measures (i.e., physiological and neuronal changes) for enhanced lie-detection. Specifically, to investigate the deception-related hemodynamic response, functional near-infrared spectroscopy (fNIRS) is applied at the prefrontal cortex besides a commercially available polygraph system. A mock crime scenario with a single-trial stimulus is set up as a deception protocol. The acquired data are classified into “true” and “lie” classes based on the fNIRS-based hemoglobin-concentration changes and polygraph-based physiological signal changes. Linear discriminant analysis is utilized as a classifier. The results indicate that the combined fNIRS-polygraph system delivers much higher classification accuracy than that of a singular system. This study demonstrates a plausible solution toward single-trial lie-detection by combining fNIRS and the polygraph.

Real-time state estimation in a flight simulator using fNIRS

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot's instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot's mental state matched significantly better than chance with the pilot's real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development.

Classification of hemodynamic responses associated with force and speed imagery for a brain-computer interface

Functional near-infrared spectroscopy (fNIRS) is an emerging optical technique, which can assess brain activities associated with tasks. In this study, six participants were asked to perform three imageries of hand clenching associated with force and speed, respectively. Joint mutual information (JMI) criterion was used to extract the optimal features of hemodynamic responses. And extreme learning machine (ELM) was employed to be the classifier. ELM solved the major bottleneck of feedforward neural networks in learning speed, this classifier was easily implemented and less sensitive to specified parameters. The 2-class fNIRS-BCI system was firstly built with an average accuracy of 76.7%, when all force and speed tasks were categorized as one class, respectively. The multi-class systems based on different levels of force and speed attempted to be investigated, the accuracies were moderate. This study provided a novel paradigm for establishing fNIRS-BCI system, and provided a possibility to produce more degrees of freedom in BCI system.