A computational model for driver's cognitive state, visual perception and intermittent attention in a distracted car following task

Resolving uncertainty on the fly: modeling adaptive driving behavior as active inference

Understanding adaptive human driving behavior, in particular how drivers manage uncertainty, is of key importance for developing simulated human driver models that can be used in the evaluation and development of autonomous vehicles. However, existing traffic psychology models of adaptive driving behavior either lack computational rigor or only address specific scenarios and/or behavioral phenomena. While models developed in the fields of machine learning and robotics can effectively learn adaptive driving behavior from data, due to their black box nature, they offer little or no explanation of the mechanisms underlying the adaptive behavior. Thus, generalizable, interpretable, computational models of adaptive human driving behavior are still rare. This paper proposes such a model based on active inference, a behavioral modeling framework originating in computational neuroscience. The model offers a principled solution to how humans trade progress against caution through policy selection based on the single mandate to minimize expected free energy. This casts goal-seeking and information-seeking (uncertainty-resolving) behavior under a single objective function, allowing the model to seamlessly resolve uncertainty as a means to obtain its goals. We apply the model in two apparently disparate driving scenarios that require managing uncertainty, (1) driving past an occluding object and (2) visual time-sharing between driving and a secondary task, and show how human-like adaptive driving behavior emerges from the single principle of expected free energy minimization.

A model for naturalistic glance behavior around Tesla Autopilot disengagements

OBJECTIVE

We present a model for visual behavior that can simulate the glance pattern observed around driver-initiated, non-critical disengagements of Tesla's Autopilot (AP) in naturalistic highway driving.

BACKGROUND

Drivers may become inattentive when using partially-automated driving systems. The safety effects associated with inattention are unknown until we have a quantitative reference on how visual behavior changes with automation.

METHODS

The model is based on glance data from 290 human initiated AP disengagement epochs. Glance duration and transition were modelled with Bayesian Generalized Linear Mixed models.

RESULTS

The model replicates the observed glance pattern across drivers. The model's components show that off-road glances were longer with AP active than without and that their frequency characteristics changed. Driving-related off-road glances were less frequent with AP active than in manual driving, while non-driving related glances to the down/center-stack areas were the most frequent and the longest (22% of the glances exceeded 2 s). Little difference was found in on-road glance duration.

CONCLUSION

Visual behavior patterns change before and after AP disengagement. Before disengagement, drivers looked less on road and focused more on non-driving related areas compared to after the transition to manual driving. The higher proportion of off-road glances before disengagement to manual driving were not compensated by longer glances ahead.

APPLICATION

The model can be used as a reference for safety assessment or to formulate design targets for driver management systems.

Inattention and Uncertainty in the Predictive Brain

Negative effects of inattention on task performance can be seen in many contexts of society and human behavior, such as traffic, work, and sports. In traffic, inattention is one of the most frequently cited causal factors in accidents. In order to identify inattention and mitigate its negative effects, there is a need for quantifying attentional demands of dynamic tasks, with a credible basis in cognitive modeling and neuroscience. Recent developments in cognitive science have led to theories of cognition suggesting that brains are an advanced prediction engine. The function of this prediction engine is to support perception and action by continuously matching incoming sensory input with top-down predictions of the input, generated by hierarchical models of the statistical regularities and causal relationships in the world. Based on the capacity of this predictive processing framework to explain various mental phenomena and neural data, we suggest it also provides a plausible theoretical and neural basis for modeling attentional demand and attentional capacity "in the wild" in terms of uncertainty and prediction error. We outline a predictive processing approach to the study of attentional demand and inattention in driving, based on neurologically-inspired theories of uncertainty processing and experimental research combining brain imaging, visual occlusion and computational modeling. A proper understanding of uncertainty processing would enable comparison of driver's uncertainty to a normative level of appropriate uncertainty, and thereby improve definition and detection of inattentive driving. This is the necessary first step toward applications such as attention monitoring systems for conventional and semi-automated driving.

A Review of Occlusion as a Tool to Assess Attentional Demand in Driving

OBJECTIVE

The aim of this review is to identify how visual occlusion contributes to our understanding of attentional demand and spare visual capacity in driving and the strengths and limitations of the method.

BACKGROUND

The occlusion technique was developed by John W. Senders to evaluate the attentional demand of driving. Despite its utility, it has been used infrequently in driver attention/inattention research.

METHOD

Visual occlusion studies in driving published between 1967 and 2020 were reviewed. The focus was on original studies in which the forward visual field was intermittently occluded while the participant was driving.

RESULTS

Occlusion studies have shown that attentional demand varies across situations and drivers and have indicated environmental, situational, and inter-individual factors behind the variability. The occlusion technique complements eye tracking in being able to indicate the temporal requirements for and redundancy in visual information sampling. The proper selection of occlusion settings depends on the target of the research.

CONCLUSION

Although there are a number of occlusion studies looking at various aspects of attentional demand, we are still only beginning to understand how these demands vary, interact, and covary in naturalistic driving.

APPLICATION

The findings of this review have methodological and theoretical implications for human factors research and for the development of distraction monitoring and in-vehicle system testing. Distraction detection algorithms and testing guidelines should consider the variability in drivers' situational and individual spare visual capacity.

Drivers use active gaze to monitor waypoints during automated driving

Automated vehicles (AVs) will change the role of the driver, from actively controlling the vehicle to primarily monitoring it. Removing the driver from the control loop could fundamentally change the way that drivers sample visual information from the scene, and in particular, alter the gaze patterns generated when under AV control. To better understand how automation affects gaze patterns this experiment used tightly controlled experimental conditions with a series of transitions from 'Manual' control to 'Automated' vehicle control. Automated trials were produced using either a 'Replay' of the driver's own steering trajectories or standard 'Stock' trials that were identical for all participants. Gaze patterns produced during Manual and Automated conditions were recorded and compared. Overall the gaze patterns across conditions were very similar, but detailed analysis shows that drivers looked slightly further ahead (increased gaze time headway) during Automation with only small differences between Stock and Replay trials. A novel mixture modelling method decomposed gaze patterns into two distinct categories and revealed that the gaze time headway increased during Automation. Further analyses revealed that while there was a general shift to look further ahead (and fixate the bend entry earlier) when under automated vehicle control, similar waypoint-tracking gaze patterns were produced during Manual driving and Automation. The consistency of gaze patterns across driving modes suggests that active-gaze models (developed for manual driving) might be useful for monitoring driver engagement during Automated driving, with deviations in gaze behaviour from what would be expected during manual control potentially indicating that a driver is not closely monitoring the automated system.

On the Difference Between Necessary and Unnecessary Glances Away From the Forward Roadway: An Occlusion Study on the Motorway

OBJECTIVE

The present study strove to distinguish traffic-related glances away from the forward roadway from non-traffic-related glances while assessing the minimum amount of visual information intake necessary for safe driving in particular scenarios.

BACKGROUND

Published gaze-based distraction detection algorithms and guidelines for distraction prevention essentially measure the time spent looking away from the forward roadway, without incorporating situation-based attentional requirements. Incorporating situation-based attentional requirements would entail an approach that not only considers the time spent looking elsewhere but also checks whether all necessary information has been sampled.

METHOD

We assess the visual sampling requirements for the forward view based on 25 experienced drivers' self-paced visual occlusion in real motorway traffic, dependent on a combination of situational factors, and compare these with their corresponding glance behavior in baseline driving.

RESULTS

Occlusion durations were on average 3 times longer than glances away from the forward roadway, and they varied substantially depending on particular maneuvers and on the proximity of other traffic, showing that interactions with nearby traffic increase perceived uncertainty. The frequency of glances away from the forward roadway was relatively stable across proximity levels and maneuvers, being very similar to what has been found in naturalistic driving.

CONCLUSION

Glances away from the forward roadway proved qualitatively different from occlusions in both their duration and when they occur. Our findings indicate that glancing away from the forward roadway for driving purposes is not the same as glancing away for other purposes, and that neither is necessarily equivalent to distraction.

Correlation Among Behavior, Personality, and Electroencephalography Revealed by a Simulated Driving Experiment

Drivers play the most important role in the human-vehicle-environment system and driving behaviors are significantly influenced by the cognitive state of the driver and his/her personality. In this paper, we aimed to explore the correlation among driving behaviors, personality and electroencephalography (EEG) using a simulated driving experiment. A total of 36 healthy subjects participated in the study. The 64-channel EEG data and the driving data, including the real-time position of the vehicle, the rotation angle of the steering wheel and the speed were acquired simultaneously during driving. The Cattell 16 Personality Factor Questionnaire (16PF) was utilized to evaluate the personalities of subjects. Through hierarchical clustering of the 16PF personality traits, the subjects were divided into four groups, i.e., the Inapprehension group, Insensitivity group, Apprehension group and the Unreasoning group, named after their representative personality trait. Their driving performance and turning behaviors were compared and EEG preprocessing, source reconstruction and the comparisons among the four groups were performed using Statistical Parameter Mapping (SPM). The turning process of the subjects can be formulated into two steps, rotating the steering wheel toward the turning direction and entering the turn, and then rotating the steering wheel back and leaving the turn. The bilateral frontal gyrus was found to be activated when turning left and right, which might be associated with its function in attention, decision-making and executive control functions in visual-spatial and visual-motor processes. The Unreasoning group had the worst driving performance with highest rates of car collision and the most intensive driving action, which was related to a higher load of visual spatial attention and decision making, when the occipital and superior frontal areas played a very important role. Apprehension (O) and Tension (Q4) had a positive correlation, and Reasoning (B) had a negative correlation with dangerous driving behaviors. Our results demonstrated the close correlation among driving behaviors, personality and EEG and may be taken as a reference for the prediction and precaution of dangerous driving behaviors in people with specific personality traits.