Can arousing feedback rectify lapses in driving? Prediction from EEG power spectra

EEG Information Transfer Changes in Different Daily Fatigue Levels During Drowsy Driving

A significant issue for traffic safety has been drowsy driving for decades. A number of studies have investigated the effects of acute fatigue on spectral power; and recent research has revealed that drowsy driving is associated with a variety of brain connections in a specific cortico-cortical pathway. In spite of this, it is still unclear how different brain regions are connected in drowsy driving at different levels of daily fatigue. This study identified the brain connectivity-behavior relationship among three different daily fatigue levels (low-, median- and high-fatigue) with the EEG data transfer entropy. According to the results, only low- and medium-fatigue groups demonstrated an inverted U-shaped change in connectivity from high performance to poor behavioral performance. In addition, from low- to high-fatigue groups, connectivity magnitude decreased in the frontal region and increased in the occipital region. These study results suggest that brain connectivity and driving behavior would be affected by different levels of daily fatigue.

Modulating Driver Alertness via Ambient Olfactory Stimulation: A Wearable Electroencephalography Study

Poor alertness levels and related changes in cognitive efficiency are common when performing monotonous tasks such as extended driving. Recent studies have investigated driver alertness decrement and possible strategies for modulating alertness with the goal of improving reaction times to safety critical events. However, most studies rely on subjective measures in assessing alertness changes, while the use of olfactory stimuli, which are known to be strong modulators of cognitive states, has not been commensurately explored in driving alertness settings. To address this gap, in the present study we investigated the effectiveness of olfactory stimuli in modulating the alertness state of drivers and explored the utility of electroencephalography (EEG) in developing objective brain-based tools for assessing the resulting changes in cortical activity. Olfactory stimulation induced a significant differential effect on braking reaction time. The corresponding effect to the cortical activity was characterized using EEG-derived metrics and the devised machine learning framework yielded a high discriminating accuracy (92.1%). Furthermore, neural activity in the alpha frequency band was found to be significantly associated with the observed drivers' behavioral changes. Overall, our results demonstrate the potential of olfactory stimuli to modulate the alertness state and the efficiency of EEG in objectively assessing the resulting cognitive changes.

Constructing the behavioral sequence of the takeover process-TOR, behavior characteristics and phases division: A real vehicle experiment

Autonomous driving will still use human-machine co-driving to handle complex situations for a long term, which requires the driver to control the vehicle and avoid hazards by executing appropriate behavioral sequences after takeover prompts. Previous studies focused on the division of static behavioral indicators and major phases in the initial phase of takeover, while lacking the construction of behavioral sequences based on the dynamic changes of behavioral characteristics during the takeover process. This study divides the takeover process in a detailed manner and investigates the impact of audio types on the behavioral sequence at each phase. 20 professional drivers performed the NDRT in autonomous driving mode on real roads, and after receiving audio prompts, they took over the vehicle and performed hazard avoidance maneuvers. The results show that the behavioral characteristics could construct the behavioral sequence of different phases, with the dynamic characteristics of the takeover operation change. In addition, different types of audio prompts will affect the timing of the takeover operation and its driving performance. Choosing different audio prompts or combinations can help improve the effect of taking over the vehicle. This study helps to provide guidance on the design of human-machine interaction for behavior optimization at different phases, so that guiding the driver to take over the vehicle safely and effectively.

Electrophysiological Correlates of Response Time in a Vigilant Attention Task

Early detection of a deficit in vigilant attention can allow for user notification or intervention. In this paper, Electrophysiological correlates of vigilant attention from a random-dot motion task were explored. Using only frontal (Fz) and parietal (Pz) EEG channels, spectral features of response time were determined. Notably, significant differ-ences in high-beta, gamma and alpha frequency bands were found between fast and slow reaction times. These results are interpreted in line with the relevant literature on arousal, off-task thought and active visuospatial attentional suppression. The presence of response-locked time-domain features was analysed. However, motor-related features obfuscated these features.

Flexible graphene/GO electrode for gel-free EEG

Objective.Developments in electroencephalography (EEG) technology have allowed the use of the brain-computer interface (BCI) outside dedicated labratories. In order to achieve long-term monitoring and detection of EEG signals for BCI application, dry electrodes with good signal quality and high bio compatibility are essential. In 2016, we proposed a flexible dry electrode made of silicone gel and Ag flakes, which showed good signal quality and mechanical robustness. However, the Ag components used in our previous design made the electrode too expensive for commercial adaptation.Approach.In this study, we developed an affordable dry electrode made of silicone gel, metal flakes and graphene/GO based on our previous design. Two types of electrodes with different graphene/GO proportions were produced to explore how the amount of graphene/GO affects the electrode.Main results.During our tests, the electrodes showed low impedance and had good signal correlation to conventional wet electrodes in both the time and frequency domains. The graphene/GO electrode also showed good signal quality in eyes-open EEG recording. We also found that the electrode with more graphene/GO had an uneven surface and worse signal quality. This suggests that adding too much graphene/GO may reduce the electrods' performance. Furthermore, we tested the proposed dry electrodes' capability in detecting steady state visually evoked potential. We found that the dry electrodes can reliably detect evoked potential changes even in the hairy occipital area.Significance.Our results showed that the proposed electrode has good signal quality and is ready for BCI applications.

Exploration of Brain Connectivity during Human Inhibitory Control Using Inter-Trial Coherence

Inhibitory control is a cognitive process that inhibits a response. It is used in everyday activities, such as driving a motorcycle, driving a car and playing a game. The effect of this process can be compared to the red traffic light in the real world. In this study, we investigated brain connectivity under human inhibitory control using the phase lag index and inter-trial coherence (ITC). The human brain connectivity gives a more accurate representation of the functional neural network. Results of electroencephalography (EEG), the data sets were generated from twelve healthy subjects during left and right hand inhibitions using the auditory stop-signal task, showed that the inter-trial coherence in delta (1-4 Hz) and theta (4-7 Hz) band powers increased over the frontal and temporal lobe of the brain. These EEG delta and theta band activities neural markers have been related to human inhibition in the frontal lobe. In addition, inter-trial coherence in the delta-theta and alpha (8-12 Hz) band powers increased at the occipital lobe through visual stimulation. Moreover, the highest brain connectivity was observed under inhibitory control in the frontal lobe between F3-F4 channels compared to temporal and occipital lobes. The greater EEG coherence and phase lag index in the frontal lobe is associated with the human response inhibition. These findings revealed new insights to understand the neural network of brain connectivity and underlying mechanisms during human response inhibition.

The effects of different fatigue levels on brain-behavior relationships in driving

BACKGROUND

In the past decade, fatigue has been regarded as one of the main factors impairing task performance and increasing behavioral lapses during driving, even leading to fatal car crashes. Although previous studies have explored the impact of acute fatigue through electroencephalography (EEG) signals, it is still unclear how different fatigue levels affect brain-behavior relationships.

METHODS

A longitudinal study was performed to investigate the brain dynamics and behavioral changes in individuals under different fatigue levels by a sustained attention task. This study used questionnaires in combination with actigraphy, a noninvasive means of monitoring human physiological activity cycles, to conduct longitudinal assessment and tracking of the objective and subjective fatigue levels of recruited participants. In this study, degrees of effectiveness score (fatigue rating) are divided into three levels (normal, reduced, and high risk) by the SAFTE fatigue model.

RESULTS

Results showed that those objective and subjective indicators were negatively correlated to behavioral performance. In addition, increased response times were accompanied by increased alpha and theta power in most brain regions, especially the posterior regions. In particular, the theta and alpha power dramatically increased in the high-fatigue (high-risk) group. Additionally, the alpha power of the occipital regions showed an inverted U-shaped change.

CONCLUSION

Our results help to explain the inconsistent findings among existing studies, which considered the effects of only acute fatigue on driving performance while ignoring different levels of resident fatigue, and potentially lead to practical and precise biomathematical models to better predict the performance of human operators.

Classification of Movement Preparation Between Attended and Distracted Self-Paced Motor Tasks

OBJECTIVE

Brain-computer interface (BCI) systems aim to control external devices by using brain signals. The performance of these systems is influenced by the user's mental state, such as attention. In this study, we classified two attention states to a target task (attended and distracted task level) while attention to the task is altered by one of three types of distractors.

METHODS

A total of 27 participants were allocated into three experimental groups and exposed to one type of distractor. An attended condition that was the same across the three groups comprised only the main task execution (self-paced dorsiflexion) while the distracted condition was concurrent execution of the main task and an oddball task (dual-task condition). Electroencephalography signals were recorded from 28 electrodes to classify the two attention states of attended or distracted task conditions by extracting temporal and spectral features.

RESULTS

The results showed that the ensemble classification accuracy using the combination of temporal and spectral features (spectro-temporal features, 82.3 ± 2.7%) was greater than using temporal (69 ± 2.2%) and spectral (80.3 ± 2.6%) features separately. The classification accuracy was computed using a combination of different channel locations, and it was demonstrated that a combination of parietal and centrally located channels was superior for classification of two attention states during movement preparation (parietal channels: 84.6 ± 1.3%, central and parietal channels: 87.2 ± 1.5%).

CONCLUSION

It is possible to monitor the users' attention to the task for different types of distractors.

SIGNIFICANCE

It has implications for online BCI systems where the requirement is for high accuracy of intention detection.

Brain Electrodynamic and Hemodynamic Signatures Against Fatigue During Driving

Fatigue is likely to be gradually cumulated in a prolonged and attention-demanding task that may adversely affect task performance. To address the brain dynamics during a driving task, this study recruited 16 subjects to participate in an event-related lane-departure driving experiment. Each subject was instructed to maintain attention and task performance throughout an hour-long driving experiment. The subjects' brain electrodynamics and hemodynamics were simultaneously recorded via 32-channel electroencephalography (EEG) and 8-source/16-detector functional near-infrared spectroscopy (fNIRS). The behavior performance demonstrated that all subjects were able to promptly respond to lane-deviation events, even if the sign of fatigue arose in the brain, which suggests that the subjects were fighting fatigue during the driving experiment. The EEG event-related analysis showed strengthening alpha suppression in the occipital cortex, a common brain region of fatigue. Furthermore, we noted increasing oxygenated hemoglobin (HbO) of the brain to fight driving fatigue in the frontal cortex, primary motor cortex, parieto-occipital cortex and supplementary motor area. In conclusion, the increasing neural activity and cortical activations were aimed at maintaining driving performance when fatigue emerged. The electrodynamic and hemodynamic signatures of fatigue fighting contribute to our understanding of the brain dynamics of driving fatigue and address driving safety issues through the maintenance of attention and behavioral performance.

Detecting and Quantifying Mind Wandering during Simulated Driving

Mind wandering is a pervasive threat to transportation safety, potentially accounting for a substantial number of crashes and fatalities. In the current study, mind wandering was induced through completion of the same task for 5 days, consisting of a 20-min monotonous freeway-driving scenario, a cognitive depletion task, and a repetition of the 20-min driving scenario driven in the reverse direction. Participants were periodically probed with auditory tones to self-report whether they were mind wandering or focused on the driving task. Self-reported mind wandering frequency was high, and did not statistically change over days of participation. For measures of driving performance, participant labeled periods of mind wandering were associated with reduced speed and reduced lane variability, in comparison to periods of on task performance. For measures of electrophysiology, periods of mind wandering were associated with increased power in the alpha band of the electroencephalogram (EEG), as well as a reduction in the magnitude of the P3a component of the event related potential (ERP) in response to the auditory probe. Results support that mind wandering has an impact on driving performance and the associated change in driver’s attentional state is detectable in underlying brain physiology. Further, results suggest that detecting the internal cognitive state of humans is possible in a continuous task such as automobile driving. Identifying periods of likely mind wandering could serve as a useful research tool for assessment of driver attention, and could potentially lead to future in-vehicle safety countermeasures.

Sustained Attention in Real Classroom Settings: An EEG Study

Sustained attention is a process that enables the maintenance of response persistence and continuous effort over extended periods of time. Performing attention-related tasks in real life involves the need to ignore a variety of distractions and inhibit attention shifts to irrelevant activities. This study investigates electroencephalography (EEG) spectral changes during a sustained attention task within a real classroom environment. Eighteen healthy students were instructed to recognize as fast as possible special visual targets that were displayed during regular university lectures. Sorting their EEG spectra with respect to response times, which indicated the level of visual alertness to randomly introduced visual stimuli, revealed significant changes in the brain oscillation patterns. The results of power-frequency analysis demonstrated a relationship between variations in the EEG spectral dynamics and impaired performance in the sustained attention task. Across subjects and sessions, prolongation of the response time was preceded by an increase in the delta and theta EEG powers over the occipital region, and decrease in the beta power over the occipital and temporal regions. Meanwhile, implementation of the complex attention task paradigm into a real-world classroom setting makes it possible to investigate specific mutual links between brain activities and factors that cause impaired behavioral performance, such as development and manifestation of classroom mental fatigue. The findings of the study set a basis for developing a system capable of estimating the level of visual attention during real classroom activities by monitoring changes in the EEG spectra.

EEG in the classroom: Synchronised neural recordings during video presentation

We performed simultaneous recordings of electroencephalography (EEG) from multiple students in a classroom, and measured the inter-subject correlation (ISC) of activity evoked by a common video stimulus. The neural reliability, as quantified by ISC, has been linked to engagement and attentional modulation in earlier studies that used high-grade equipment in laboratory settings. Here we reproduce many of the results from these studies using portable low-cost equipment, focusing on the robustness of using ISC for subjects experiencing naturalistic stimuli. The present data shows that stimulus-evoked neural responses, known to be modulated by attention, can be tracked for groups of students with synchronized EEG acquisition. This is a step towards real-time inference of engagement in the classroom.

Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data

Investigations of the neuro-physiological correlates of mental loads, or states, have attracted significant attention recently, as it is particularly important to evaluate mental fatigue in drivers operating a motor vehicle. In this research, we collected multimodal EEG/ECG/EOG and fNIRS data simultaneously to develop algorithms to explore neuro-physiological correlates of drivers' mental states. Each subject performed simulated driving under two different conditions (well-rested and sleep-deprived) on different days. During the experiment, we used 68 electrodes for EEG/ECG/EOG and 8 channels for fNIRS recordings. We extracted the prominent features of each modality to distinguish between the well-rested and sleep-deprived conditions, and all multimodal features, except EOG, were combined to quantify mental fatigue during driving. Finally, a novel driving condition level (DCL) was proposed that distinguished clearly between the features of well-rested and sleep-deprived conditions. This proposed DCL measure may be applicable to real-time monitoring of the mental states of vehicle drivers. Further, the combination of methods based on each classifier yielded substantial improvements in the classification accuracy between these two conditions.

An EEG-Based Fatigue Detection and Mitigation System

Research has indicated that fatigue is a critical factor in cognitive lapses because it negatively affects an individual’s internal state, which is then manifested physiologically. This study explores neurophysiological changes, measured by electroencephalogram (EEG), due to fatigue. This study further demonstrates the feasibility of an online closed-loop EEG-based fatigue detection and mitigation system that detects physiological change and can thereby prevent fatigue-related cognitive lapses. More importantly, this work compares the efficacy of fatigue detection and mitigation between the EEG-based and a nonEEG-based random method. Twelve healthy subjects participated in a sustained-attention driving experiment. Each participant’s EEG signal was monitored continuously and a warning was delivered in real-time to participants once the EEG signature of fatigue was detected. Study results indicate suppression of the alpha- and theta-power of an occipital component and improved behavioral performance following a warning signal; these findings are in line with those in previous studies. However, study results also showed reduced warning efficacy (i.e. increased response times (RTs) to lane deviations) accompanied by increased alpha-power due to the fluctuation of warnings over time. Furthermore, a comparison of EEG-based and nonEEG-based random approaches clearly demonstrated the necessity of adaptive fatigue-mitigation systems, based on a subject’s cognitive level, to deliver warnings. Analytical results clearly demonstrate and validate the efficacy of this online closed-loop EEG-based fatigue detection and mitigation mechanism to identify cognitive lapses that may lead to catastrophic incidents in countless operational environments.

Mind-Wandering Tends to Occur under Low Perceptual Demands during Driving

Fluctuations in attention behind the wheel poses a significant risk for driver safety. During transient periods of inattention, drivers may shift their attention towards internally-directed thoughts or feelings at the expense of staying focused on the road. This study examined whether increasing task difficulty by manipulating involved sensory modalities as the driver detected the lane-departure in a simulated driving task would promote a shift of brain activity between different modes of processing, reflected by brain network dynamics on electroencephalographic sources. Results showed that depriving the driver of salient sensory information imposes a relatively more perceptually-demanding task, leading to a stronger activation in the task-positive network. When the vehicle motion feedback is available, the drivers may rely on vehicle motion to perceive the perturbations, which frees attentional capacity and tends to activate the default mode network. Such brain network dynamics could have major implications for understanding fluctuations in driver attention and designing advance driver assistance systems.

Developing an EEG-based on-line closed-loop lapse detection and mitigation system

In America, 60% of adults reported that they have driven a motor vehicle while feeling drowsy, and at least 15–20% of fatal car accidents are fatigue-related. This study translates previous laboratory-oriented neurophysiological research to design, develop, and test an On-line Closed-loop Lapse Detection and Mitigation (OCLDM) System featuring a mobile wireless dry-sensor EEG headgear and a cell-phone based real-time EEG processing platform. Eleven subjects participated in an event-related lane-keeping task, in which they were instructed to manipulate a randomly deviated, fixed-speed cruising car on a 4-lane highway. This was simulated in a 1st person view with an 8-screen and 8-projector immersive virtual-reality environment. When the subjects experienced lapses or failed to respond to events during the experiment, auditory warning was delivered to rectify the performance decrements. However, the arousing auditory signals were not always effective. The EEG spectra exhibited statistically significant differences between effective and ineffective arousing signals, suggesting that EEG spectra could be used as a countermeasure of the efficacy of arousing signals. In this on-line pilot study, the proposed OCLDM System was able to continuously detect EEG signatures of fatigue, deliver arousing warning to subjects suffering momentary cognitive lapses, and assess the efficacy of the warning in near real-time to rectify cognitive lapses. The on-line testing results of the OCLDM System validated the efficacy of the arousing signals in improving subjects' response times to the subsequent lane-departure events. This study may lead to a practical on-line lapse detection and mitigation system in real-world environments.