Tonic and phasic EEG and behavioral changes induced by arousing feedback

EEG-based neurophysiological indices for expert psychomotor performance - a review

A primary objective of current human neuropsychological performance research is to define the physiological correlates of adaptive knowledge utilization, in order to support the enhanced execution of both simple and complex tasks. Within the present article, electroencephalography-based neurophysiological indices characterizing expert psychomotor performance, will be explored. As a means of characterizing fundamental processes underlying efficient psychometric performance, the neural efficiency model will be evaluated in terms of alpha-wave-based selective cortical processes. Cognitive and motor domains will initially be explored independently, which will act to encapsulate the task-related neuronal adaptive requirements for enhanced psychomotor performance associating with the neural efficiency model. Moderating variables impacting the practical application of such neuropsychological model, will also be investigated. As a result, the aim of this review is to provide insight into detectable task-related modulation involved in developed neurocognitive strategies which support heightened psychomotor performance, for the implementation within practical settings requiring a high degree of expert performance (such as sports or military operational settings).

EEG Information Transfer Changes in Different Daily Fatigue Levels During Drowsy Driving

A significant issue for traffic safety has been drowsy driving for decades. A number of studies have investigated the effects of acute fatigue on spectral power; and recent research has revealed that drowsy driving is associated with a variety of brain connections in a specific cortico-cortical pathway. In spite of this, it is still unclear how different brain regions are connected in drowsy driving at different levels of daily fatigue. This study identified the brain connectivity-behavior relationship among three different daily fatigue levels (low-, median- and high-fatigue) with the EEG data transfer entropy. According to the results, only low- and medium-fatigue groups demonstrated an inverted U-shaped change in connectivity from high performance to poor behavioral performance. In addition, from low- to high-fatigue groups, connectivity magnitude decreased in the frontal region and increased in the occipital region. These study results suggest that brain connectivity and driving behavior would be affected by different levels of daily fatigue.

Brain State Relays Self-Processing and Heartbeat-Evoked Cortical Responses

The self has been proposed to be grounded in interoceptive processing, with heartbeat-evoked cortical activity as a neurophysiological marker of this processing. However, inconsistent findings have been reported on the relationship between heartbeat-evoked cortical responses and self-processing (including exteroceptive- and mental-self-processing). In this review, we examine previous research on the association between self-processing and heartbeat-evoked cortical responses and highlight the divergent temporal-spatial characteristics and brain regions involved. We propose that the brain state relays the interaction between self-processing and heartbeat-evoked cortical responses and thus accounts for the inconsistency. The brain state, spontaneous brain activity which highly and continuously changes in a nonrandom way, serves as the foundation upon which the brain functions and was proposed as a point in an extremely high-dimensional space. To elucidate our assumption, we provide reviews on the interactions between dimensions of brain state with both self-processing and heartbeat-evoked cortical responses. These interactions suggest the relay of self-processing and heartbeat-evoked cortical responses by brain state. Finally, we discuss possible approaches to investigate whether and how the brain state impacts the self-heart interaction.

Physiological and Psychological Effects of Volatile Organic Compounds from Dried Common Rush (Juncus effusus L. var. decipiens Buchen.) on Humans

This study compared the participants’ physiological responses and subjective evaluations of air scented with different concentrations of common rush (Juncus effusus L. var. decipiens Buchen.) (30 g and 15 g, with fresh air as a control). We asked 20 participants to complete a series of visual discrimination tasks while inhaling two different air samples. We evaluated (1) brain activity, (2) autonomic nervous activity, and (3) blood pressure and pulse rate, (4) in combination with self-evaluation. In addition, we quantified the concentrations of volatile organic compounds. The participants reported the scent to be sour, pungent, and smelly; this impression was likely caused by hexanal and acetic acid. Although the self-evaluations showed that participants did not enjoy the scent, their alpha amplitudes of electroencephalogram and parasympathetic nervous activity were increased, suggesting that participants were relaxed in this atmosphere. Moreover, a lower concentration resulted in a greater induction of relaxation. While the air was not pleasant-smelling, the volatile organic compounds present had a positive psychophysiological impact.

Wielding and evaluating the removal composition of common artefacts in EEG signals for driving behaviour analysis

Noninvasive EEG signals provide neural activity information at high resolution, of which human mental status can be properly detected. However, artefacts always exist in brain oscillatory EEG signals and thus impede the accuracy and reliability of relevant analysis, especially in real-world tasks. Moreover, the use of a mature artefact identification method cannot assure impeccable artefact separation; this leads to a trade-off between removing contaminated information and losing valuable information. This study addresses this problem by investigating a simulator-based driving behaviour analysis using a car-following scenario to correlate the EEG-based mental features with behavioural responses. The study develops an architecture for an artefact composition pool and proposes three integrated prediction models to evaluate the removal compositions of the EEG artefacts. Three errors (mean absolute, root mean square, mean absolute percentage) and R-squared index are considered for measuring the performance of the models. The results show that the best-performing composition outperformed the no-removal and all-removal cases by 11.75% and 4.28% improvements, respectively. Specifically, we investigate different common artefacts including eye blinks, horizontal eye movements, vertical eye movements, generic discontinuities and muscle artefacts. The gained knowledge on artefact removal, EEG spectral features and stimuli-response patterns can be further applied to properly manipulate real-world EEG signals and develop an effective brain-computer interface.

A Pilot Study on EEG-Based Evaluation of Visually Induced Motion Sickness

The most prominent problem in virtual reality (VR) technology is that users may experience motion sickness-like symptoms when they immerse into a VR environment. These symptoms are recognized as visually induced motion sickness (VIMS) or virtual reality motion sickness (VRMS). The objectives of this study were to investigate the association between the electroencephalogram (EEG) and subjectively rated VIMS level (VIMSL) and find the EEG markers for VIMS evaluation. In this study, a VR-based vehicle-driving simulator (VDS) was used to induce VIMS symptoms, and a wearable EEG device with four electrodes, the Muse, was used to collect EEG data of subjects. Our results suggest that individual tolerance, susceptibility, and recoverability to VIMS varied largely among subjects; the following markers were shown to be significantly different from no-VIMS and VIMS states (P < 0.05): (1) Means of gravity frequency (GF) for theta@FP1, alpha@TP9, alpha@FP2, alpha@TP10, and beta@FP1; (2) Standard deviation of GF for alpha@TP9, alpha@FP1, alpha@FP2, alpha@TP10, and alpha@(FP2-FP1); (3) Standard deviation of power spectral entropy (PSE) for FP1; (4) Means of Kolmogorov complexity (KC) for TP9, FP1, and FP2. These results also demonstrate that it is feasible to perform VIMS evaluation using an EEG device with a small number of electrodes.

The effects of different fatigue levels on brain-behavior relationships in driving

BACKGROUND

In the past decade, fatigue has been regarded as one of the main factors impairing task performance and increasing behavioral lapses during driving, even leading to fatal car crashes. Although previous studies have explored the impact of acute fatigue through electroencephalography (EEG) signals, it is still unclear how different fatigue levels affect brain-behavior relationships.

METHODS

A longitudinal study was performed to investigate the brain dynamics and behavioral changes in individuals under different fatigue levels by a sustained attention task. This study used questionnaires in combination with actigraphy, a noninvasive means of monitoring human physiological activity cycles, to conduct longitudinal assessment and tracking of the objective and subjective fatigue levels of recruited participants. In this study, degrees of effectiveness score (fatigue rating) are divided into three levels (normal, reduced, and high risk) by the SAFTE fatigue model.

RESULTS

Results showed that those objective and subjective indicators were negatively correlated to behavioral performance. In addition, increased response times were accompanied by increased alpha and theta power in most brain regions, especially the posterior regions. In particular, the theta and alpha power dramatically increased in the high-fatigue (high-risk) group. Additionally, the alpha power of the occipital regions showed an inverted U-shaped change.

CONCLUSION

Our results help to explain the inconsistent findings among existing studies, which considered the effects of only acute fatigue on driving performance while ignoring different levels of resident fatigue, and potentially lead to practical and precise biomathematical models to better predict the performance of human operators.

Exploring the Brain Responses to Driving Fatigue Through Simultaneous EEG and fNIRS Measurements

Fatigue is one problem with driving as it can lead to difficulties with sustaining attention, behavioral lapses, and a tendency to ignore vital information or operations. In this research, we explore multimodal physiological phenomena in response to driving fatigue through simultaneous functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) recordings with the aim of investigating the relationships between hemodynamic and electrical features and driving performance. Sixteen subjects participated in an event-related lane-deviation driving task while measuring their brain dynamics through fNIRS and EEGs. Three performance groups, classified as Optimal, Suboptimal, and Poor, were defined for comparison. From our analysis, we find that tonic variations occur before a deviation, and phasic variations occur afterward. The tonic results show an increased concentration of oxygenated hemoglobin (HbO₂) and power changes in the EEG theta, alpha, and beta bands. Both dynamics are significantly correlated with deteriorated driving performance. The phasic EEG results demonstrate event-related desynchronization associated with the onset of steering vehicle in all power bands. The concentration of phasic HbO₂ decreased as performance worsened. Further, the negative correlations between tonic EEG delta and alpha power and HbO₂ oscillations suggest that activations in HbO₂ are related to mental fatigue. In summary, combined hemodynamic and electrodynamic activities can provide complete knowledge of the brain's responses as evidence of state changes during fatigue driving.

Electroencephalographic and peripheral temperature dynamics during a prolonged psychomotor vigilance task

Attention lapses and fatigue are a main source of impaired performance that can lead to accidents. This study analyzed electroencephalographic (EEG) dynamics and body skin temperature as markers of attentional fluctuations in non-sleep deprived subjects during a 45min Psychomotor Vigilance Task (PVT). Independent Component Analysis and time-frequency analysis were used to evaluate the EEG data. Results showed a positive association between distal and distal-to-proximal gradient (DPG) temperatures and reaction time (RT); increments in EEG power in alpha-, theta- and beta-band frequencies in parieto-occipital, central-medial and frontal components, were associated with poor performance (slower RT) in the task. This generalized power increment fits with an increased activity in the default mode network, associated with attention lapses. This study highlights the potential use of the PVT as a tool to obtain individual physiological indices of vigilance and fatigue that could be applied to other vigilance tasks typically performed in occupational settings.

Multi-channel EEG recordings during a sustained-attention driving task

We describe driver behaviour and brain dynamics acquired from a 90-minute sustained-attention task in an immersive driving simulator. The data included 62 sessions of 32-channel electroencephalography (EEG) data for 27 subjects driving on a four-lane highway who were instructed to keep the car cruising in the centre of the lane. Lane-departure events were randomly induced to cause the car to drift from the original cruising lane towards the left or right lane. A complete trial included events with deviation onset, response onset, and response offset. The next trial, in which the subject was instructed to drive back to the original cruising lane, began 5-10 seconds after finishing the previous trial. We believe that this dataset will lead to the development of novel neural processing methodology that can be used to index brain cortical dynamics and detect driving fatigue and drowsiness. This publicly available dataset will be beneficial to the neuroscience and brain-computer interface communities.

EEG classification of driver mental states by deep learning

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .

Brain Network Changes in Fatigued Drivers: A Longitudinal Study in a Real-World Environment Based on the Effective Connectivity Analysis and Actigraphy Data

The analysis of neurophysiological changes during driving can clarify the mechanisms of fatigue, considered an important cause of vehicle accidents. The fluctuations in alertness can be investigated as changes in the brain network connections, reflected in the direction and magnitude of the information transferred. Those changes are induced not only by the time on task but also by the quality of sleep. In an unprecedented 5-month longitudinal study, daily sampling actigraphy and EEG data were collected during a sustained-attention driving task within a near-real-world environment. Using a performance index associated with the subjects' reaction times and a predictive score related to the sleep quality, we identify fatigue levels in drivers and investigate the shifts in their effective connectivity in different frequency bands, through the analysis of the dynamical coupling between brain areas. Study results support the hypothesis that combining EEG, behavioral and actigraphy data can reveal new features of the decline in alertness. In addition, the use of directed measures such as the Convergent Cross Mapping can contribute to the development of fatigue countermeasure devices.

Modeling brain dynamic state changes with adaptive mixture independent component analysis

There is a growing interest in neuroscience in assessing the continuous, endogenous, and nonstationary dynamics of brain network activity supporting the fluidity of human cognition and behavior. This non-stationarity may involve ever-changing formation and dissolution of active cortical sources and brain networks. However, unsupervised approaches to identify and model these changes in brain dynamics as continuous transitions between quasi-stable brain states using unlabeled, noninvasive recordings of brain activity have been limited. This study explores the use of adaptive mixture independent component analysis (AMICA) to model multichannel electroencephalographic (EEG) data with a set of ICA models, each of which decomposes an adaptively learned portion of the data into statistically independent sources. We first show that AMICA can segment simulated quasi-stationary EEG data and accurately identify ground-truth sources and source model transitions. Next, we demonstrate that AMICA decomposition, applied to 6-13 channel scalp recordings from the CAP Sleep Database, can characterize sleep stage dynamics, allowing 75% accuracy in identifying transitions between six sleep stages without use of EEG power spectra. Finally, applied to 30-channel data from subjects in a driving simulator, AMICA identifies models that account for EEG during faster and slower response to driving challenges, respectively. We show changes in relative probabilities of these models allow effective prediction of subject response speed and moment-by-moment characterization of state changes within single trials. AMICA thus provides a generic unsupervised approach to identifying and modeling changes in EEG dynamics. Applied to continuous, unlabeled multichannel data, AMICA may likely be used to detect and study any changes in cognitive states.

A subject-transfer framework for obviating inter- and intra-subject variability in EEG-based drowsiness detection

Inter- and intra-subject variability pose a major challenge to decoding human brain activity in brain-computer interfaces (BCIs) based on non-invasive electroencephalogram (EEG). Conventionally, a time-consuming and laborious training procedure is performed on each new user to collect sufficient individualized data, hindering the applications of BCIs on monitoring brain states (e.g. drowsiness) in real-world settings. This study proposes applying hierarchical clustering to assess the inter- and intra-subject variability within a large-scale dataset of EEG collected in a simulated driving task, and validates the feasibility of transferring EEG-based drowsiness-detection models across subjects. A subject-transfer framework is thus developed for detecting drowsiness based on a large-scale model pool from other subjects and a small amount of alert baseline calibration data from a new user. The model pool ensures the availability of positive model transferring, whereas the alert baseline data serve as a selector of decoding models in the pool. Compared with the conventional within-subject approach, the proposed framework remarkably reduced the required calibration time for a new user by 90% (18.00 min-1.72 ± 0.36 min) without compromising performance (p = 0.0910) when sufficient existing data are available. These findings suggest a practical pathway toward plug-and-play drowsiness detection and can ignite numerous real-world BCI applications.

Sustained Attention in Real Classroom Settings: An EEG Study

Sustained attention is a process that enables the maintenance of response persistence and continuous effort over extended periods of time. Performing attention-related tasks in real life involves the need to ignore a variety of distractions and inhibit attention shifts to irrelevant activities. This study investigates electroencephalography (EEG) spectral changes during a sustained attention task within a real classroom environment. Eighteen healthy students were instructed to recognize as fast as possible special visual targets that were displayed during regular university lectures. Sorting their EEG spectra with respect to response times, which indicated the level of visual alertness to randomly introduced visual stimuli, revealed significant changes in the brain oscillation patterns. The results of power-frequency analysis demonstrated a relationship between variations in the EEG spectral dynamics and impaired performance in the sustained attention task. Across subjects and sessions, prolongation of the response time was preceded by an increase in the delta and theta EEG powers over the occipital region, and decrease in the beta power over the occipital and temporal regions. Meanwhile, implementation of the complex attention task paradigm into a real-world classroom setting makes it possible to investigate specific mutual links between brain activities and factors that cause impaired behavioral performance, such as development and manifestation of classroom mental fatigue. The findings of the study set a basis for developing a system capable of estimating the level of visual attention during real classroom activities by monitoring changes in the EEG spectra.

Monitoring alert and drowsy states by modeling EEG source nonstationarity

OBJECTIVE

As a human brain performs various cognitive functions within ever-changing environments, states of the brain characterized by recorded brain activities such as electroencephalogram (EEG) are inevitably nonstationary. The challenges of analyzing the nonstationary EEG signals include finding neurocognitive sources that underlie different brain states and using EEG data to quantitatively assess the state changes.

APPROACH

This study hypothesizes that brain activities under different states, e.g. levels of alertness, can be modeled as distinct compositions of statistically independent sources using independent component analysis (ICA). This study presents a framework to quantitatively assess the EEG source nonstationarity and estimate levels of alertness. The framework was tested against EEG data collected from 10 subjects performing a sustained-attention task in a driving simulator.

MAIN RESULTS

Empirical results illustrate that EEG signals under alert versus drowsy states, indexed by reaction speeds to driving challenges, can be characterized by distinct ICA models. By quantifying the goodness-of-fit of each ICA model to the EEG data using the model deviation index (MDI), we found that MDIs were significantly correlated with the reaction speeds (r  =  -0.390 with alertness models and r  =  0.449 with drowsiness models) and the opposite correlations indicated that the two models accounted for sources in the alert and drowsy states, respectively. Based on the observed source nonstationarity, this study also proposes an online framework using a subject-specific ICA model trained with an initial (alert) state to track the level of alertness. For classification of alert against drowsy states, the proposed online framework achieved an averaged area-under-curve of 0.745 and compared favorably with a classic power-based approach.

SIGNIFICANCE

This ICA-based framework provides a new way to study changes of brain states and can be applied to monitoring cognitive or mental states of human operators in attention-critical settings or in passive brain-computer interfaces.

ICA-Derived EEG Correlates to Mental Fatigue, Effort, and Workload in a Realistically Simulated Air Traffic Control Task

Electroencephalograph (EEG) has been increasingly studied to identify distinct mental factors when persons perform cognitively demanding tasks. However, most of these studies examined EEG correlates at channel domain, which suffers the limitation that EEG signals are the mixture of multiple underlying neuronal sources due to the volume conduction effect. Moreover, few studies have been conducted in real-world tasks. To precisely probe EEG correlates with specific neural substrates to mental factors in real-world tasks, the present study examined EEG correlates to three mental factors, i.e., mental fatigue [also known as time-on-task (TOT) effect], workload and effort, in EEG component signals, which were obtained using an independent component analysis (ICA) on high-density EEG data. EEG data were recorded when subjects performed a realistically simulated air traffic control (ATC) task for 2 h. Five EEG independent component (IC) signals that were associated with specific neural substrates (i.e., the frontal, central medial, motor, parietal, occipital areas) were identified. Their spectral powers at their corresponding dominant bands, i.e., the theta power of the frontal IC and the alpha power of the other four ICs, were detected to be correlated to mental workload and effort levels, measured by behavioral metrics. Meanwhile, a linear regression analysis indicated that spectral powers at five ICs significantly increased with TOT. These findings indicated that different levels of mental factors can be sensitively reflected in EEG signals associated with various brain functions, including visual perception, cognitive processing, and motor outputs, in real-world tasks. These results can potentially aid in the development of efficient operational interfaces to ensure productivity and safety in ATC and beyond.

Differential Functionality of Right and Left Parietal Activity in Controlling a Motor Vehicle

Driving a motor vehicle is an inherently complex task that requires robust control to avoid catastrophic accidents. Drivers must maintain their vehicle in the middle of the travel lane to avoid high speed collisions with other traffic. Interestingly, while a vehicle’s lane deviation (LD) is critical, studies have demonstrated that heading error (HE) is one of the primary variables drivers use to determine a steering response, which directly controls the position of the vehicle in the lane. In this study, we examined how the brain represents the dichotomy between control/response parameters (heading, reaction time (RT), and steering wheel corrections) and task-critical parameters (LD). Specifically, we examined electroencephalography (EEG) alpha band power (8–13 Hz) from estimated sources in right and left parietal regions, and related this activity to four metrics of driving performance. Our results demonstrate differential task involvement between the two hemispheres: right parietal activity was most closely related to LD, whereas left parietal activity was most closely related to HE, RT and steering responses. Furthermore, HE, RT and steering wheel corrections increased over the duration of the experiment while LD did not. Collectively, our results suggest that the brain uses differential monitoring and control strategies in the right and left parietal regions to control a motor vehicle. Our results suggest that the regulation of this control changes over time while maintaining critical task performance. These results are interpreted in two complementary theoretical frameworks: the uncontrolled manifold and compensatory control theories. The central tenet of these frameworks permits performance variability in parameters (i.e., HE, RT and steering) so far as it does not interfere with critical task execution (i.e., LD). Our results extend the existing research by demonstrating potential neural substrates for this phenomenon which may serve as potential targets for brain-computer interfaces that predict poor driving performance.

Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data

Investigations of the neuro-physiological correlates of mental loads, or states, have attracted significant attention recently, as it is particularly important to evaluate mental fatigue in drivers operating a motor vehicle. In this research, we collected multimodal EEG/ECG/EOG and fNIRS data simultaneously to develop algorithms to explore neuro-physiological correlates of drivers' mental states. Each subject performed simulated driving under two different conditions (well-rested and sleep-deprived) on different days. During the experiment, we used 68 electrodes for EEG/ECG/EOG and 8 channels for fNIRS recordings. We extracted the prominent features of each modality to distinguish between the well-rested and sleep-deprived conditions, and all multimodal features, except EOG, were combined to quantify mental fatigue during driving. Finally, a novel driving condition level (DCL) was proposed that distinguished clearly between the features of well-rested and sleep-deprived conditions. This proposed DCL measure may be applicable to real-time monitoring of the mental states of vehicle drivers. Further, the combination of methods based on each classifier yielded substantial improvements in the classification accuracy between these two conditions.

An EEG-Based Fatigue Detection and Mitigation System

Research has indicated that fatigue is a critical factor in cognitive lapses because it negatively affects an individual’s internal state, which is then manifested physiologically. This study explores neurophysiological changes, measured by electroencephalogram (EEG), due to fatigue. This study further demonstrates the feasibility of an online closed-loop EEG-based fatigue detection and mitigation system that detects physiological change and can thereby prevent fatigue-related cognitive lapses. More importantly, this work compares the efficacy of fatigue detection and mitigation between the EEG-based and a nonEEG-based random method. Twelve healthy subjects participated in a sustained-attention driving experiment. Each participant’s EEG signal was monitored continuously and a warning was delivered in real-time to participants once the EEG signature of fatigue was detected. Study results indicate suppression of the alpha- and theta-power of an occipital component and improved behavioral performance following a warning signal; these findings are in line with those in previous studies. However, study results also showed reduced warning efficacy (i.e. increased response times (RTs) to lane deviations) accompanied by increased alpha-power due to the fluctuation of warnings over time. Furthermore, a comparison of EEG-based and nonEEG-based random approaches clearly demonstrated the necessity of adaptive fatigue-mitigation systems, based on a subject’s cognitive level, to deliver warnings. Analytical results clearly demonstrate and validate the efficacy of this online closed-loop EEG-based fatigue detection and mitigation mechanism to identify cognitive lapses that may lead to catastrophic incidents in countless operational environments.

Mind-Wandering Tends to Occur under Low Perceptual Demands during Driving

Fluctuations in attention behind the wheel poses a significant risk for driver safety. During transient periods of inattention, drivers may shift their attention towards internally-directed thoughts or feelings at the expense of staying focused on the road. This study examined whether increasing task difficulty by manipulating involved sensory modalities as the driver detected the lane-departure in a simulated driving task would promote a shift of brain activity between different modes of processing, reflected by brain network dynamics on electroencephalographic sources. Results showed that depriving the driver of salient sensory information imposes a relatively more perceptually-demanding task, leading to a stronger activation in the task-positive network. When the vehicle motion feedback is available, the drivers may rely on vehicle motion to perceive the perturbations, which frees attentional capacity and tends to activate the default mode network. Such brain network dynamics could have major implications for understanding fluctuations in driver attention and designing advance driver assistance systems.

Automatic design for independent component analysis based brain-computer interfacing

This study proposes a new framework, independent component ensemble, to leverage the acquired knowledge into a truly automatic and on-line EEG-based brain-computer interfacing (BCI). The envisioned design includes: (1) independent source recover using independent component analysis (ICA) (2) automatic selection of the independent components of interest (ICi) associated with human behaviors; (3) multiple classifiers with a parallel constructing and processing structure; and (4) a simple fusion scheme to combine the decisions from multiple classifiers. Its implications in BCI are demonstrated through a sample application: cognitive-state monitoring of participants performing a realistic sustained-attention driving task. Empirical results showed the proposed ensemble design could provide an improvement of 7% ~ 15% in overall accuracy for the classification of the arousal state and the driving performance. In summary, constructing ICi-ensemble classifiers and combining their outputs demonstrates a practical option for ICA-based BCIs to reduce the risk of not obtaining any desired independent source or selecting an inadequate component. Most importantly, the ensemble design for integrating information across multiple brain areas creates potentials for developing more complicated BCIs for real world applications.

Novel measure of driver and vehicle interaction demonstrates transient changes related to alerting

Driver behavior and vehicle-road kinematics have been shown to change over prolonged periods of driving; however, the interaction between these two indices has not been examined. Here we develop a measure that examines how drivers turn the steering wheel relative to heading error velocity, which the authors call the relative steering wheel compensation (RSWC). The RSWC transiently changes on a short time scale coincident with a verbal query embedded within the study paradigm. In contrast, more traditional variables are dynamic over longer time scales consistent with previous research. The results suggest drivers alter their behavioral output (steering wheel correction) relative to sensory input (vehicle heading error velocity) on a distinct temporal scale and may reflect an interaction of alerting and control.

Developing an EEG-based on-line closed-loop lapse detection and mitigation system

In America, 60% of adults reported that they have driven a motor vehicle while feeling drowsy, and at least 15–20% of fatal car accidents are fatigue-related. This study translates previous laboratory-oriented neurophysiological research to design, develop, and test an On-line Closed-loop Lapse Detection and Mitigation (OCLDM) System featuring a mobile wireless dry-sensor EEG headgear and a cell-phone based real-time EEG processing platform. Eleven subjects participated in an event-related lane-keeping task, in which they were instructed to manipulate a randomly deviated, fixed-speed cruising car on a 4-lane highway. This was simulated in a 1st person view with an 8-screen and 8-projector immersive virtual-reality environment. When the subjects experienced lapses or failed to respond to events during the experiment, auditory warning was delivered to rectify the performance decrements. However, the arousing auditory signals were not always effective. The EEG spectra exhibited statistically significant differences between effective and ineffective arousing signals, suggesting that EEG spectra could be used as a countermeasure of the efficacy of arousing signals. In this on-line pilot study, the proposed OCLDM System was able to continuously detect EEG signatures of fatigue, deliver arousing warning to subjects suffering momentary cognitive lapses, and assess the efficacy of the warning in near real-time to rectify cognitive lapses. The on-line testing results of the OCLDM System validated the efficacy of the arousing signals in improving subjects' response times to the subsequent lane-departure events. This study may lead to a practical on-line lapse detection and mitigation system in real-world environments.

Identification of temporal variations in mental workload using locally-linear-embedding-based EEG feature reduction and support-vector-machine-based clustering and classification techniques

Identifying the abnormal changes of mental workload (MWL) over time is quite crucial for preventing the accidents due to cognitive overload and inattention of human operators in safety-critical human-machine systems. It is known that various neuroimaging technologies can be used to identify the MWL variations. In order to classify MWL into a few discrete levels using representative MWL indicators and small-sized training samples, a novel EEG-based approach by combining locally linear embedding (LLE), support vector clustering (SVC) and support vector data description (SVDD) techniques is proposed and evaluated by using the experimentally measured data. The MWL indicators from different cortical regions are first elicited by using the LLE technique. Then, the SVC approach is used to find the clusters of these MWL indicators and thereby to detect MWL variations. It is shown that the clusters can be interpreted as the binary class MWL. Furthermore, a trained binary SVDD classifier is shown to be capable of detecting slight variations of those indicators. By combining the two schemes, a SVC-SVDD framework is proposed, where the clear-cut (smaller) cluster is detected by SVC first and then a subsequent SVDD model is utilized to divide the overlapped (larger) cluster into two classes. Finally, three-class MWL levels (low, normal and high) can be identified automatically. The experimental data analysis results are compared with those of several existing methods. It has been demonstrated that the proposed framework can lead to acceptable computational accuracy and has the advantages of both unsupervised and supervised training strategies.

Estimating endogenous changes in task performance from EEG

Brain wave activity is known to correlate with decrements in behavioral performance as individuals enter states of fatigue, boredom, or low alertness.Many BCI technologies are adversely affected by these changes in user state, limiting their application and constraining their use to relatively short temporal epochs where behavioral performance is likely to be stable. Incorporating a passive BCI that detects when the user is performing poorly at a primary task, and adapts accordingly may prove to increase overall user performance. Here, we explore the potential for extending an established method to generate continuous estimates of behavioral performance from ongoing neural activity; evaluating the extended method by applying it to the original task domain, simulated driving; and generalizing the method by applying it to a BCI-relevant perceptual discrimination task. Specifically, we used EEG log power spectra and sequential forward floating selection (SFFS) to estimate endogenous changes in behavior in both a simulated driving task and a perceptual discrimination task. For the driving task the average correlation coefficient between the actual and estimated lane deviation was 0.37 ± 0.22 (μ ± σ). For the perceptual discrimination task we generated estimates of accuracy, reaction time, and button press duration for each participant. The correlation coefficients between the actual and estimated behavior were similar for these three metrics (accuracy = 0.25 ± 0.37, reaction time = 0.33 ± 0.23, button press duration = 0.36 ± 0.30). These findings illustrate the potential for modeling time-on-task decrements in performance from concurrent measures of neural activity.

Wireless and wearable EEG system for evaluating driver vigilance

Brain activity associated with attention sustained on the task of safe driving has received considerable attention recently in many neurophysiological studies. Those investigations have also accurately estimated shifts in drivers' levels of arousal, fatigue, and vigilance, as evidenced by variations in their task performance, by evaluating electroencephalographic (EEG) changes. However, monitoring the neurophysiological activities of automobile drivers poses a major measurement challenge when using a laboratory-oriented biosensor technology. This work presents a novel dry EEG sensor based mobile wireless EEG system (referred to herein as Mindo) to monitor in real time a driver's vigilance status in order to link the fluctuation of driving performance with changes in brain activities. The proposed Mindo system incorporates the use of a wireless and wearable EEG device to record EEG signals from hairy regions of the driver conveniently. Additionally, the proposed system can process EEG recordings and translate them into the vigilance level. The study compares the system performance between different regression models. Moreover, the proposed system is implemented using JAVA programming language as a mobile application for online analysis. A case study involving 15 study participants assigned a 90 min sustained-attention driving task in an immersive virtual driving environment demonstrates the reliability of the proposed system. Consistent with previous studies, power spectral analysis results confirm that the EEG activities correlate well with the variations in vigilance. Furthermore, the proposed system demonstrated the feasibility of predicting the driver's vigilance in real time.

Kinesthesia in a sustained-attention driving task

This study investigated the effects of kinesthetic stimuli on brain activities during a sustained-attention task in an immersive driving simulator. Tonic and phasic brain responses on multiple timescales were analyzed using time-frequency analysis of electroencephalographic (EEG) sources identified by independent component analysis (ICA). Sorting EEG spectra with respect to reaction times (RT) to randomly introduced lane-departure events revealed distinct effects of kinesthetic stimuli on the brain under different performance levels. Experimental results indicated that EEG spectral dynamics highly correlated with performance lapses when driving involved kinesthetic feedback. Furthermore, in the realistic environment involving both visual and kinesthetic feedback, a transitive relationship of power spectra between optimal-, suboptimal-, and poor-performance groups was found predominately across most of the independent components. In contrast to the static environment with visual input only, kinesthetic feedback reduced theta-power augmentation in the central and frontal components when preparing for action and error monitoring, while strengthening alpha suppression in the central component while steering the wheel. In terms of behavior, subjects tended to have a short response time to process unexpected events with the assistance of kinesthesia, yet only when their performance was optimal. Decrease in attentional demand, facilitated by kinesthetic feedback, eventually significantly increased the reaction time in the suboptimal-performance state. Neurophysiological evidence of mutual relationships between behavioral performance and neurocognition in complex task paradigms and experimental environments, presented in this study, might elucidate our understanding of distributed brain dynamics, supporting natural human cognition and complex coordinated, multi-joint naturalistic behavior, and lead to improved understanding of brain-behavior relations in operating environments.

Can arousing feedback rectify lapses in driving? Prediction from EEG power spectra

OBJECTIVE

This study explores the neurophysiological changes, measured using an electroencephalogram (EEG), in response to an arousing warning signal delivered to drowsy drivers, and predicts the efficacy of the feedback based on changes in the EEG.

APPROACH

Eleven healthy subjects participated in sustained-attention driving experiments. The driving task required participants to maintain their cruising position and compensate for randomly induced lane deviations using the steering wheel, while their EEG and driving performance were continuously monitored. The arousing warning signal was delivered to participants who experienced momentary behavioral lapses, failing to respond rapidly to lane-departure events (specifically the reaction time exceeded three times the alert reaction time).

MAIN RESULTS

The results of our previous studies revealed that arousing feedback immediately reversed deteriorating driving performance, which was accompanied by concurrent EEG theta- and alpha-power suppression in the bilateral occipital areas. This study further proposes a feedback efficacy assessment system to accurately estimate the efficacy of arousing warning signals delivered to drowsy participants by monitoring the changes in their EEG power spectra immediately thereafter. The classification accuracy was up 77.8% for determining the need for triggering additional warning signals.

SIGNIFICANCE

The findings of this study, in conjunction with previous studies on EEG correlates of behavioral lapses, might lead to a practical closed-loop system to predict, monitor and rectify behavioral lapses of human operators in attention-critical settings.

Preventing lapse in performance using a drowsiness monitoring and management system

Research on public security, especially the safe manipulation and control of vehicles, has gained increasing attention in recent years. This study proposes a closed-loop drowsiness monitoring and management system that can estimate subjects' driving performance. The system observes electroencephalographic (EEG) dynamics and behavioral changes, delivers arousing feedback to individuals experiencing momentary cognitive lapses, and assesses the efficacy of the feedback. Results of this study showed that the arousing feedback immediately improved subject performance, which was accompanied by concurrent theta- and alpha-power suppression in the bilateral occipital areas. This study further demonstrated the feasibility of accurately assessing the efficacy of arousing feedback presented to drowsy participants by monitoring the changes in their EEG power spectra.

Losing the struggle to stay awake: divergent thalamic and cortical activity during microsleeps

Maintaining alertness is critical for safe and successful performance of most human activities. Consequently, microsleeps during continuous visuomotor tasks, such as driving, can be very serious, not only disrupting performance but sometimes leading to injury or death due to accidents. We have investigated the neural activity underlying behavioral microsleeps--brief (0.5-15 s) episodes of complete failure to respond accompanied by slow eye-closures--and EEG theta activity during drowsiness in a continuous task. Twenty healthy normally-rested participants performed a 50-min continuous tracking task while fMRI, EEG, eye-video, and responses were simultaneously recorded. Visual rating of performance and eye-video revealed that 70% of the participants had frequent microsleeps. fMRI analysis revealed a transient decrease in thalamic, posterior cingulate, and occipital cortex activity and an increase in frontal, posterior parietal, and parahippocampal activity during microsleeps. The transient activity was modulated by the duration of the microsleep. In subjects with frequent microsleeps, power in the post-central EEG theta was positively correlated with the BOLD signal in the thalamus, basal forebrain, and visual, posterior parietal, and prefrontal cortices. These results provide evidence for distinct neural changes associated with microsleeps and with EEG theta activity during drowsiness in a continuous task. They also suggest that the occurrence of microsleeps during an active task is not a global deactivation process but involves localized activation of fronto-parietal cortex, which, despite a transient loss of arousal, may constitute a mechanism by which these regions try to restore responsiveness.

A collaborative brain-computer interface for improving human performance

Electroencephalogram (EEG) based brain-computer interfaces (BCI) have been studied since the 1970s. Currently, the main focus of BCI research lies on the clinical use, which aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. However, the BCI technology can also be used to improve human performance for normal healthy users. Although this application has been proposed for a long time, little progress has been made in real-world practices due to technical limits of EEG. To overcome the bottleneck of low single-user BCI performance, this study proposes a collaborative paradigm to improve overall BCI performance by integrating information from multiple users. To test the feasibility of a collaborative BCI, this study quantitatively compares the classification accuracies of collaborative and single-user BCI applied to the EEG data collected from 20 subjects in a movement-planning experiment. This study also explores three different methods for fusing and analyzing EEG data from multiple subjects: (1) Event-related potentials (ERP) averaging, (2) Feature concatenating, and (3) Voting. In a demonstration system using the Voting method, the classification accuracy of predicting movement directions (reaching left vs. reaching right) was enhanced substantially from 66% to 80%, 88%, 93%, and 95% as the numbers of subjects increased from 1 to 5, 10, 15, and 20, respectively. Furthermore, the decision of reaching direction could be made around 100–250 ms earlier than the subject's actual motor response by decoding the ERP activities arising mainly from the posterior parietal cortex (PPC), which are related to the processing of visuomotor transmission. Taken together, these results suggest that a collaborative BCI can effectively fuse brain activities of a group of people to improve the overall performance of natural human behavior.

Inter-individual variability of oscillatory responses to subject's own name. A single-subject analysis

In previous studies event-related potentials and oscillations in response to subject's own name have been analyzed extensively on group-level in healthy subjects and in patients with a disorder of consciousness. Subject's own name as a deviant produces a P3. With equiprobable stimuli, non-phase-locked alpha oscillations are smaller in response to subject's own name compared to other names or subject's own name backwards. However, little is known about replicability on a single-subject level. Seventeen healthy subjects were assessed in an own-name paradigm with equiprobable stimuli of subject's own name, another name, and subject's own name backwards. Event-related potentials and non-phase locked oscillations were analyzed with single-subject, non-parametric statistics. No consistent results were found either for ERPs or for the non-phase locked changes of oscillatory activities. Only 4 subjects showed a robust effect as expected, that is, a lower activity in the alpha-beta range to subject's own name compared to other conditions. Four subjects elicited a higher activity for subject's own name. Thus, analyzing the EEG reactivity in the own-name paradigm with equiprobable stimuli on a single-subject level yields a high variance between subjects. In future research, single-subject statistics should be applied for examining the validity of physiologic measurements in other paradigms and for examining the pattern of reactivity in patients.

Spatial and temporal EEG dynamics of dual-task driving performance

Background

Driver distraction is a significant cause of traffic accidents. The aim of this study is to investigate Electroencephalography (EEG) dynamics in relation to distraction during driving. To study human cognition under a specific driving task, simulated real driving using virtual reality (VR)-based simulation and designed dual-task events are built, which include unexpected car deviations and mathematics questions.

Methods

We designed five cases with different stimulus onset asynchrony (SOA) to investigate the distraction effects between the deviations and equations. The EEG channel signals are first converted into separated brain sources by independent component analysis (ICA). Then, event-related spectral perturbation (ERSP) changes of the EEG power spectrum are used to evaluate brain dynamics in time-frequency domains.

Results

Power increases in the theta and beta bands are observed in relation with distraction effects in the frontal cortex. In the motor area, alpha and beta power suppressions are also observed. All of the above results are consistently observed across 15 subjects. Additionally, further analysis demonstrates that response time and multiple cortical EEG power both changed significantly with different SOA.

Conclusions

This study suggests that theta power increases in the frontal area is related to driver distraction and represents the strength of distraction in real-life situations.

Arousing feedback rectifies lapse in performance and corresponding EEG power spectrum

This study explores electroencephalographic (EEG) dynamics and behavioral changes in response to arousing auditory signals presented to individuals experiencing momentary cognitive lapses. Arousing auditory feedback was delivered to the subjects in half of the non-responded lane-deviation events during a sustained-attention driving task, which immediately agitated subject's responses to the events. The improved behavioral performance was accompanied by concurrent power suppression in the theta- and alpha-bands in the lateral occipital cortices. This study further explores the feasibility of estimating the efficacy of arousing feedback presented to the drowsy subjects by monitoring the changes in EEG power spectra.