Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

Investigating multilevel cognitive processing within error-free and error-prone feedback conditions in executed and observed car driving

Accident analyses repeatedly reported the considerable contribution of run-off-road incidents to fatalities in road traffic, and despite considerable advances in assistive technologies to mitigate devastating consequences, little insight into the drivers' brain response during such accident scenarios has been gained. While various literature documents neural correlates to steering motion, the driver's mental state, and the impact of distraction and fatigue on driving performance, the cortical substrate of continuous deviations of a car from the road - i.e., how the brain represents a varying discrepancy between the intended and observed car position and subsequently assigns customized levels of corrective measures - remains unclear. Furthermore, the superposition of multiple subprocesses, such as visual and erroneous feedback processing, performance monitoring, or motor control, complicates a clear interpretation of engaged brain regions within car driving tasks. In the present study, we thus attempted to disentangle these subprocesses, employing passive and active steering conditions within both error-free and error-prone vehicle operation conditions. We recorded EEG signals of 26 participants in 13 sessions, simultaneously measuring pairs of Executors (actively steering) and Observers (strictly observing) during a car driving task. We observed common brain patterns in the Executors regardless of error-free or error-prone vehicle operation, albeit with a shift in spectral activity from motor beta to occipital alpha oscillations within erroneous conditions. Further, significant frontocentral differences between Observers and Executors, tracing back to the caudal anterior cingulate cortex, arose during active steering conditions, indicating increased levels of motor-behavioral cognitive control. Finally, we present regression results of both the steering signal and the car position, indicating that a regression of continuous deviations from the road utilizing the EEG might be feasible.

Identification of Optimal and Most Significant Event Related Brain Functional Network

Advancements in network science have facilitated the study of brain communication networks. Existing techniques for identifying event-related brain functional networks (BFNs) often result in fully connected networks. However, determining the optimal and most significant network representation for event-related BFNs is crucial for understanding complex brain networks. The presence of both false and genuine connections in the fully connected network requires network thresholding to eliminate false connections. However, a generalized framework for thresholding in network neuroscience is currently lacking. To address this, we propose four novel methods that leverage network properties, energy, and efficiency to select a generalized threshold level. This threshold serves as the basis for identifying the optimal and most significant event-related BFN. We validate our methods on an openly available emotion dataset and demonstrate their effectiveness in identifying multiple events. Our proposed approach can serve as a versatile thresholding technique to represent the fully connected network as an event-related BFN.

Pupil-linked arousal correlates with neural activity prior to sensorimotor decisions

Objective.Sensorimotor decisions require the brain to process external information and combine it with relevant knowledge prior to actions. In this study, we explore the neural predictors of motor actions in a novel, realistic driving task designed to study decisions while driving.Approach.Through a spatiospectral assessment of functional connectivity during the premotor period, we identified the organization of visual cortex regions of interest into a distinct scene processing network. Additionally, we identified a motor action selection network characterized by coherence between the anterior cingulate cortex (ACC) and dorsolateral prefrontal cortex (DLPFC).Main results.We show that steering behavior can be predicted from oscillatory power in the visual cortex, DLPFC, and ACC. Power during the premotor periods (specific to the theta and beta bands) correlates with pupil-linked arousal and saccade duration.Significance.We interpret our findings in the context of network-level correlations with saccade-related behavior and show that the DLPFC is a key node in arousal circuitry and in sensorimotor decisions.

Blink-related EEG activity measures cognitive load during proactive and reactive driving

Assessing drivers' cognitive load is crucial for driving safety in challenging situations. This research employed the occurrence of drivers' natural eye blinks as cues in continuously recorded EEG data to assess the cognitive workload while reactive or proactive driving. Twenty-eight participants performed either a lane-keeping task with varying levels of crosswind (reactive) or curve road (proactive). The blink event-related potentials (bERPs) and spectral perturbations (bERSPs) were analyzed to assess cognitive load variations. The study found that task load during reactive driving did not significantly impact bERPs or bERSPs, possibly due to enduring alertness for vehicle control. The proactive driving revealed significant differences in the occipital N1 component with task load, indicating the necessity to adapt the attentional resources allocation based on road demands. Also, increased steering complexity led to decreased frontal N2, parietal P3, occipital P2 amplitudes, and alpha power, requiring more cognitive resources for processing relevant information. Interestingly, the proactive and reactive driving scenarios demonstrated a significant interaction at the parietal P2 and occipital N1 for three difficulty levels. The study reveals that EEG measures related to natural eye blink behavior provide insights into the effect of cognitive load on different driving tasks, with implications for driver safety.

Neurophysiological Predictors of Self-Reported Difficulty in a Virtual-Reality Driving Scenario

Our perception of subjective difficulty in complex tasks, such as driving, is a judgment that is likely a result of dynamic interactions between distributed brain regions. In this paper, we investigate how neurophysiological markers associated with arousal state are informative of this perceived difficulty throughout a driving task. We do this by classifying subjective difficulty reports of subjects using set of features that include neural, autonomic, and eye behavioral markers. We subsequently assess the importance of these features in the classification. We find that though multiple EEG linked to cognitive control and, motor performance linked to classification of subjective difficulty, only pupil diameter, a measure of pupil-linked arousal, is strongly linked to both measured self-reported difficulty and actual task performance. We interpret our findings in the context of arousal pathways influencing performance and discuss their relevance to future brain-computer interface systems.

EEG-EMG coupling as a hybrid method for steering detection in car driving settings

Understanding mental processes in complex human behavior is a key issue in driving, representing a milestone for developing user-centered assistive driving devices. Here, we propose a hybrid method based on electroencephalographic (EEG) and electromyographic (EMG) signatures to distinguish left and right steering in driving scenarios. Twenty-four participants took part in the experiment consisting of recordings of 128-channel EEG and EMG activity from deltoids and forearm extensors in non-ecological and ecological steering tasks. Specifically, we identified the EEG mu rhythm modulation correlates with motor preparation of self-paced steering actions in the non-ecological task, while the concurrent EMG activity of the left (right) deltoids correlates with right (left) steering. Consequently, we exploited the mu rhythm de-synchronization resulting from the non-ecological task to detect the steering side using cross-correlation analysis with the ecological EMG signals. Results returned significant cross-correlation values showing the coupling between the non-ecological EEG feature and the muscular activity collected in ecological driving conditions. Moreover, such cross-correlation patterns discriminate the steering side earlier relative to the single EMG signal. This hybrid system overcomes the limitation of the EEG signals collected in ecological settings such as low reliability, accuracy, and adaptability, thus adding to the EMG the characteristic predictive power of the cerebral data. These results prove how it is possible to complement different physiological signals to control the level of assistance needed by the driver.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-021-09776-w.

Directed Brain Network Analysis for Fatigue Driving Based on EEG Source Signals

Fatigue driving is one of the major factors that leads to traffic accidents. Long-term monotonous driving can easily cause a decrease in the driver's attention and vigilance, manifesting a fatigue effect. This paper proposes a means of revealing the effects of driving fatigue on the brain's information processing abilities, from the aspect of a directed brain network based on electroencephalogram (EEG) source signals. Based on current source density (CSD) data derived from EEG signals using source analysis, a directed brain network for fatigue driving was constructed by using a directed transfer function. As driving time increased, the average clustering coefficient as well as the average path length gradually increased; meanwhile, global efficiency gradually decreased for most rhythms, suggesting that deep driving fatigue enhances the brain's local information integration abilities while weakening its global abilities. Furthermore, causal flow analysis showed electrodes with significant differences between the awake state and the driving fatigue state, which were mainly distributed in several areas of the anterior and posterior regions, especially under the theta rhythm. It was also found that the ability of the anterior regions to receive information from the posterior regions became significantly worse in the driving fatigue state. These findings may provide a theoretical basis for revealing the underlying neural mechanisms of driving fatigue.

A pilot study to understand the relationship between cortical arousals and leg movements during sleep

Leg movements during sleep occur in patients with sleep pathology and healthy individuals. Some (but not all) leg movements during sleep are related to cortical arousals which occur without conscious awareness but have a significant effect of sleep fragmentation. Detecting leg movements during sleep that are associated with cortical arousals can provide unique insight into the nature and quality of sleep. In this study, a novel leg movement monitor that uses a unique capacitive displacement sensor and 6-axis inertial measurement unit, is used in conjunction with polysomnography to understand the relationship between leg movement and electroencephalogram (EEG) defined cortical arousals. In an approach that we call neuro-extremity analysis, directed connectivity metrics are used to interrogate causal linkages between EEG and leg movements measured by the leg movement sensors. The capacitive displacement measures were more closely related to EEG-defined cortical arousals than inertial measurements. Second, the neuro-extremity analysis reveals a temporally evolving connectivity pattern that is consistent with a model of cortical arousals in which brainstem dysfunction leads to near-instantaneous leg movements and a delayed, filtered signal to the cortex leading to the cortical arousal during sleep.

Predictive models for human-AI nexus in group decision making

Machine learning (ML) and artificial intelligence (AI) have had a profound impact on our lives. Domains like health and learning are naturally helped by human-AI interactions and decision making. In these areas, as ML algorithms prove their value in making important decisions, humans add their distinctive expertise and judgment on social and interpersonal issues that need to be considered in tandem with algorithmic inputs of information. Some questions naturally arise. What rules and regulations should be invoked on the employment of AI, and what protocols should be in place to evaluate available AI resources? What are the forms of effective communication and coordination with AI that best promote effective human-AI teamwork? In this review, we highlight factors that we believe are especially important in assembling and managing human-AI decision making in a group setting.

Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of various types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. At the macro scale, bibliometric aspects of these studies are analysed. At the micro scale, different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or neural activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Potential topics in driving behaviour research are identified that could benefit from the adoption of neuroimaging methods in future studies. In terms of practicality, while fMRI and MEG experiments have proven rather invasive and technologically challenging for adoption in driving behaviour research, EEG and fNIRS applications have been more diverse. They have even been tested beyond simulated driving settings, in field driving experiments. Advantages and limitations of each of these four neuroimaging methods in the context of driving behaviour experiments are outlined in the paper.

Driving Style Recognition Based on Electroencephalography Data From a Simulated Driving Experiment

Driving style is a very important indicator and a crucial measurement of a driver's performance and ability to drive in a safe and protective manner. A dangerous driving style would possibly result in dangerous behaviors. If the driving styles can be recognized by some appropriate classification methods, much attention could be paid to the drivers with dangerous driving styles. The driving style recognition module can be integrated into the advanced driving assistance system (ADAS), which integrates different modules to improve driving automation, safety and comfort, and then the driving safety could be enhanced by pre-warning the drivers or adjusting the vehicle's controlling parameters when the dangerous driving style is detected. In most previous studies, driver's questionnaire data and vehicle's objective driving data were utilized to recognize driving styles. And promising results were obtained. However, these methods were indirect or subjective in driving style evaluation. In this paper a method based on objective driving data and electroencephalography (EEG) data was presented to classify driving styles. A simulated driving system was constructed and the EEG data and the objective driving data were collected synchronously during the simulated driving. The driving style of each participant was classified by clustering the driving data via K-means. Then the EEG data was denoised and the amplitude and the Power Spectral Density (PSD) of four frequency bands were extracted as the EEG features by Fast Fourier transform and Welch. Finally, the EEG features, combined with the classification results of the driving data were used to train a Support Vector Machine (SVM) model and a leave-one-subject-out cross validation was utilized to evaluate the performance. The SVM classification accuracy was about 80.0%. Conservative drivers showed higher PSDs in the parietal and occipital areas in the alpha and beta bands, aggressive drivers showed higher PSD in the temporal area in the delta and theta bands. These results imply that different driving styles were related with different driving strategies and mental states and suggest the feasibility of driving style recognition from EEG patterns.

A Novel Framework for Estimating Time-Varying Multivariate Autoregressive Models and Application to Cardiovascular Responses to Acute Exercise

OBJECTIVE

We present a novel modeling framework for identifying time-varying (TV) couplings between time-series of biomedical relevance.

METHODS

The proposed methodology is based on multivariate autoregressive (MVAR) models, which have been extensively used to study couplings between biosignals. Contrary to the standard estimation methods that assume time-invariant relationships, we propose a modified recursive Kalman filter (KF) to track changes in the model parameters. We perform model order selection and hyperparameter optimization simultaneously using Genetic Algorithms, greatly improving accuracy and computation time. In addition, we address the effect of residual heteroscedasticity, possibly associated with event-related changes or phase transitions during a given experimental protocol, on the TV-MVAR coupling measures by using Generalized Autoregressive Conditional Heteroskedasticity (GARCH) models to fit the TV-MVAR residuals.

RESULTS

Using simulated data, we show that the proposed framework yields more accurate parameter estimates compared to the conventional KF, particularly when the true system parameters exhibit different rate of variations over time. Furthermore, by accounting for heteroskedasticity, we obtain more accurate estimates of the strength and directionality of the underlying couplings. We also use our approach to investigate TV hemodynamic interactions during exercise in young and old healthy adults, as well as individuals with chronic stroke. We extract TV coupling patterns that reflect well known exercise-induced effects on the underlying regulatory mechanisms with excellent time resolution.

CONCLUSION AND SIGNIFICANCE

The proposed modeling framework can be used to efficiently quantify TV couplings between biosignals. It is fully automated and does not require prior knowledge of the system TV characteristics.

Global brain dynamics during social exclusion predict subsequent behavioral conformity

Individuals react differently to social experiences; for example, people who are more sensitive to negative social experiences, such as being excluded, may be more likely to adapt their behavior to fit in with others. We examined whether functional brain connectivity during social exclusion in the fMRI scanner can be used to predict subsequent conformity to peer norms. Adolescent males (n = 57) completed a two-part study on teen driving risk: a social exclusion task (Cyberball) during an fMRI session and a subsequent driving simulator session in which they drove alone and in the presence of a peer who expressed risk-averse or risk-accepting driving norms. We computed the difference in functional connectivity between social exclusion and social inclusion from each node in the brain to nodes in two brain networks, one previously associated with mentalizing (medial prefrontal cortex, temporoparietal junction, precuneus, temporal poles) and another with social pain (dorsal anterior cingulate cortex, anterior insula). Using predictive modeling, this measure of global connectivity during exclusion predicted the extent of conformity to peer pressure during driving in the subsequent experimental session. These findings extend our understanding of how global neural dynamics guide social behavior, revealing functional network activity that captures individual differences.

Data-driven brain network models differentiate variability across language tasks

The relationship between brain structure and function has been probed using a variety of approaches, but how the underlying structural connectivity of the human brain drives behavior is far from understood. To investigate the effect of anatomical brain organization on human task performance, we use a data-driven computational modeling approach and explore the functional effects of naturally occurring structural differences in brain networks. We construct personalized brain network models by combining anatomical connectivity estimated from diffusion spectrum imaging of individual subjects with a nonlinear model of brain dynamics. By performing computational experiments in which we measure the excitability of the global brain network and spread of synchronization following a targeted computational stimulation, we quantify how individual variation in the underlying connectivity impacts both local and global brain dynamics. We further relate the computational results to individual variability in the subjects' performance of three language-demanding tasks both before and after transcranial magnetic stimulation to the left-inferior frontal gyrus. Our results show that task performance correlates with either local or global measures of functional activity, depending on the complexity of the task. By emphasizing differences in the underlying structural connectivity, our model serves as a powerful tool to assess individual differences in task performances, to dissociate the effect of targeted stimulation in tasks that differ in cognitive demand, and to pave the way for the development of personalized therapeutics.

Age-Related Differences in Pro-active Driving Behavior Revealed by EEG Measures

Healthy aging is associated with a decline in cognitive functions. This may become an issue when complex tasks have to be performed like driving a car in a demanding traffic situation. On the other hand, older people are able to compensate for age-related deficits, e.g., by deploying extra mental effort and other compensatory strategies. The present study investigated the interplay of age, task workload, and mental effort using EEG measures and a proactive driving task, in which 16 younger and 16 older participants had to keep a virtual car on track on a curvy road. Total oscillatory power and relative power in Theta and Alpha bands were analyzed, as well as event-related potentials (ERPs) to task-irrelevant regular and irregular sound stimuli. Steering variability and Theta power increased with increasing task load (i.e., with shaper bends of the road), while Alpha power decreased. This pattern of workload and mental effort was found in both age groups. However, only in the older group a relationship between steering variability and Theta power occurred: better steering performance was associated with higher Theta power, reflecting higher mental effort. Higher Theta power while driving was also associated with a stronger increase in reported subjective fatigue in the older group. In the younger group, lower steering variability came along with lower ERP responses to deviant sound stimuli, reflecting reduced processing of task-irrelevant environmental stimuli. In sum, better performance in proactive driving (i.e., more alert steering behavior) was associated with increased mental effort in the older group, and higher attentional focus on the task in the younger group, indicating age-specific strategies in the way younger and older drivers manage demanding (driving) tasks.

Time-varying MVAR algorithms for directed connectivity analysis: Critical comparison in simulations and benchmark EEG data

Human brain function depends on directed interactions between multiple areas that evolve in the subsecond range. Time-varying multivariate autoregressive (tvMVAR) modeling has been proposed as a way to help quantify directed functional connectivity strengths with high temporal resolution. While several tvMVAR approaches are currently available, there is a lack of unbiased systematic comparative analyses of their performance and of their sensitivity to parameter choices. Here, we critically compare four recursive tvMVAR algorithms and assess their performance while systematically varying adaptation coefficients, model order, and signal sampling rate. We also compared two ways of exploiting repeated observations: single-trial modeling followed by averaging, and multi-trial modeling where one tvMVAR model is fitted across all trials. Results from numerical simulations and from benchmark EEG recordings showed that: i) across a broad range of model orders all algorithms correctly reproduced patterns of interactions; ii) signal downsampling degraded connectivity estimation accuracy for most algorithms, although in some cases downsampling was shown to reduce variability in the estimates by lowering the number of parameters in the model; iii) single-trial modeling followed by averaging showed optimal performance with larger adaptation coefficients than previously suggested, and showed slower adaptation speeds than multi-trial modeling. Overall, our findings identify strengths and weaknesses of existing tvMVAR approaches and provide practical recommendations for their application to modeling dynamic directed interactions from electrophysiological signals.

Evaluating Pro- and Re-Active Driving Behavior by Means of the EEG

Traffic safety essentially depends on the drivers' alertness and vigilance, especially in monotonous or demanding driving situations. Brain oscillatory EEG activity offers insight into a drivers' mental state and has therefore attracted much attention in the past. However, EEG measures do not only vary with internal factors like attentional engagement and vigilance but might also interact with external factors like time on task, task demands, or the degree to which a traffic situation is predictable. In order to identify EEG parameters for cognitive mechanisms involved in tasks of high and low controllability, the present study investigated the interaction of time on task, task load, and cognitive controllability in simulated driving scenarios, using an either re-active or pro-active driving task. Participants performed a lane-keeping task, half of them compensating varying levels of crosswind (re-active task), and the other half driving along a winding road (pro-active task). Both driving tasks were adjusted with respect to difficulty. The analysis of oscillatory EEG parameters showed an increase in total power (1-30 Hz) with time on task, with decreasing task load, and in the re-active compared to the pro-active task. Furthermore, the relative power in Alpha band increased with decreasing task load and time on task, while relative Theta power showed the opposite pattern. Moreover, relative Alpha power was also higher in the re-active, than pro-active, driving situation, an effect that even increased with time on task. The results demonstrate that the controllability of a driving situation has a similar effect on oscillatory EEG activity like time on task and task load.

Proactive vs. reactive car driving: EEG evidence for different driving strategies of older drivers

Aging is associated with a large heterogeneity in the extent of age-related changes in sensory, motor, and cognitive functions. All these functions can influence the performance in complex tasks like car driving. The present study aims to identify potential differences in underlying cognitive processes that may explain inter-individual variability in driving performance. Younger and older participants performed a one-hour monotonous driving task in a driving simulator under varying crosswind conditions, while behavioral and electrophysiological data were recorded. Overall, younger and older drivers showed comparable driving performance (lane keeping). However, there was a large difference in driving lane variability within the older group. Dividing the older group in two subgroups with low vs. high driving lane variability revealed differences between the two groups in electrophysiological correlates of mental workload, consumption of mental resources, and activation and sustaining of attention: Older drivers with high driving lane variability showed higher frontal Alpha and Theta activity than older drivers with low driving lane variability and—with increasing crosswind—a more pronounced decrease in Beta activity. These results suggest differences in driving strategies of older and younger drivers, with the older drivers using either a rather proactive and alert driving strategy (indicated by low driving lane variability and lower Alpha and Beta activity), or a rather reactive strategy (indicated by high driving lane variability and higher Alpha activity).