Glass half-full: On-road glance metrics differentiate crashes from near-crashes in the 100-Car data

Exploratory Development of Algorithms for Determining Driver Attention Status

OBJECTIVE

Varying driver distraction algorithms were developed using vehicle kinematics and driver gaze data obtained from a camera-based driver monitoring system (DMS).

BACKGROUND

Distracted driving characteristics can be difficult to accurately detect due to wide variation in driver behavior across driving environments. The growing availability of information about drivers and their involvement in the driving task increases the opportunity for accurately recognizing attention state.

METHOD

A baseline for driver distraction levels was developed using a video feed of 24 separate drivers in varying naturalistic driving conditions. This initial assessment was used to develop four buffer-based algorithms that aimed to determine a driver's real-time attentiveness, via a variety of metrics and combinations thereof.

RESULTS

Of those tested, the optimal algorithm included ungrouped glance locations and speed. Notably, as an algorithm's performance of detecting very distracted drivers improved, its accuracy for correctly identifying attentive drivers decreased.

CONCLUSION

At a minimum, drivers' gaze position and vehicle speed should be included when designing driver distraction algorithms to delineate between glance patterns observed at high and low speeds. Distraction algorithms should be designed with an understanding of their limitations, including instances in which they may fail to detect distracted drivers, or falsely notify attentive drivers.

APPLICATION

This research adds to the body of knowledge related to driver distraction and contributes to available methods to potentially address and reduce occurrences. Machine learning algorithms can build on the data elements discussed to increase distraction detection accuracy using robust artificial intelligence.

Anticipatory Driving in Automated Vehicles: The Effects of Driving Experience and Distraction

OBJECTIVE

To understand the influence of driving experience and distraction on drivers' anticipation of upcoming traffic events in automated vehicles.

BACKGROUND

In nonautomated vehicles, experienced drivers spend more time looking at cues that indicate upcoming traffic events compared with novices, and distracted drivers spend less time looking at these cues compared with nondistracted drivers. Further, pre-event actions (i.e., proactive control actions prior to traffic events) are more prevalent among experienced drivers and nondistracted drivers. However, there is a research gap on the combined effects of experience and distraction on driver anticipation in automated vehicles.

METHODS

A simulator experiment was conducted with 16 experienced and 16 novice drivers in a vehicle equipped with adaptive cruise control and lane-keeping assist systems (resulting in SAE Level 2 driving automation). Half of the participants in each experience group were provided with a self-paced primarily visual-manual secondary task.

RESULTS

Drivers with the task spent less time looking at cues and were less likely to perform anticipatory driving behaviors (i.e., pre-event actions or preparation for pre-event actions such as hovering fingers over the automation disengage button). Experienced drivers exhibited more anticipatory driving behaviors, but their attention toward the cues was similar to novices for both task conditions.

CONCLUSION

In line with nonautomated vehicle research, in automated vehicles, secondary task engagement impedes anticipation while driving experience facilitates anticipation.

APPLICATION

Though Level 2 automation can relieve drivers of manually controlling the vehicle and allow engagement in distractions, visual-manual distraction engagement can impede anticipatory driving and should be restricted.

On the relationship between occlusion times and in-car glance durations in simulated driving

Drivers have spare visual capacity in driving, and often this capacity is used for engaging in secondary in-car tasks. Previous research has suggested that the spare visual capacity could be estimated with the occlusion method. However, the relationship between drivers' occlusion times and in-car glance duration preferences has not been sufficiently investigated for granting occlusion times the role of an estimate of spare visual capacity. We conducted a driving simulator experiment (N = 30) and investigated if there is an association between drivers' occlusion times and in-car glance durations in a given driving scenario. Furthermore, we explored which factors and variables could explain the strength of the association. The findings suggest an association between occlusion time preferences and in-car glance durations in visually and cognitively low demanding unstructured tasks but that this association is lost if the in-car task is more demanding. The findings might be explained by the inability to utilize peripheral vision for lane-keeping when conducting in-car tasks and/or by in-car task structures that override drivers' preferences for the in-car glance durations. It seems that the occlusion technique could be utilized as an estimate of drivers' spare visual capacity in research - but with caution. It is strongly recommended to use occlusion times in combination with driving performance metrics. There is less spare visual capacity if this capacity is used for secondary tasks that interfere with the driver's ability to utilize peripheral vision for driving or preferences for the in-car glance durations. However, we suggest that the occlusion method can be a valid method to control for inter-individual differences in in-car glance duration preferences when investigating the visual distraction potential of, for instance, in-vehicle infotainment systems.

Repeated conditionally automated driving on the road: How do drivers leave the loop over time?

The goal of this on the road driving study was to investigate how drivers adapt their behavior when driving with conditional vehicle automation (SAE L3) on different occasions. Specifically, we focused on changes in how fast drivers took over control from automation and how their gaze off the road changed over time. On each of three consecutive days, 21 participants drove for 50 min, in a conditionally automated vehicle (Wizard of Oz methodology), on a typical German commuting highway. Over these rides the take-over behavior and gaze behavior were analyzed. The data show that drivers' reactions to non-critical, system initiated, take-overs took about 5.62 s and did not change within individual rides, but on average became 0.72 s faster over the three rides. After these self-paced take-over requests a final urgent take-over request was issued at the end of the third ride. In this scenario participants took over rapidly with an average of 5.28 s. This urgent take-over time was not found to be different from the self-paced take-over requests in the same ride. Regarding gaze behavior, participants' overall longest glance off the road and the percentage of time looked off the road increased within each ride, but stayed stable over the three rides. Taken together, our results suggest that drivers regularly leave the loop by gazing off the road, but multiple exposures to take-over situations in automated driving allow drivers to come back into loop faster.

Impact of Temporary Browsing Restrictions on Drivers’ Situation Awareness When Interacting with In-Vehicle Infotainment Systems

Looking away from the road during a task degrades situation awareness of potential hazards. Long glances back to the road rebuild this awareness and are thought to be critical for maintaining good vehicle control and recognizing conflicts. To further investigate the importance of rebuilding situation awareness, a controlled test-track study was performed that evaluated drivers’ hazard awareness and response performance to a surprise event after completing a task that involved pausing partway through it to look back at the road. Thirty-two drivers completed a visual-manual infotainment system secondary task. Half of the drivers were instructed to pause their browsing mid-task, while the others were not. While the task was being performed, a lead vehicle activated its hazard lights. It then unexpectedly dropped a fake muffler once drivers completed the task. Drivers’ visual attention to the road and their ability to respond to the muffler were measured. The drivers that paused their browsing were more aware of the lead vehicle’s hazard lights, showed less surprise to the dropped muffler, and executed more measured avoidance maneuvers compared to the drivers that did not pause their browsing. These findings suggest that drivers’ situation awareness can be better maintained when task interactions are paced, allowing for longer monitoring of the environment. Mechanisms that encourage drivers to take restorative on-road glances during extended browsing may be a key aspect of an overall approach to mitigating driver distraction.

Survival analysis of the unsafe behaviors leading to urban expressway crashes

A common cause of vehicle crashes on urban expressways lies in the unsafe behaviors of drivers. This study focused on analyzing the influence of various unsafe behaviors on crash duration. Based on actual video image of vehicle crashes, 14 unsafe behaviors were identified for the analysis of crashes on urban expressways. Using the correspondence analysis method, the correlation among unsafe behaviors and collision types was obtained. Nonparametric survival analysis was then presented to obtain the survival rate curves of sideswipe crashes and rear-end crashes. Finally, parametric survival analysis method can get the influence of unsafe behaviors on crash duration. The survival rate of any time was quantified through the reasoning of key unsafe behaviors for different types of crashes. The results show that there were striking differences in the duration among different types of crashes. The unsafe behaviors had a significant impact on duration for different types of crashes. This study focused on the duration under the influence of unsafe behaviors before the crash, and the results provide valuable information to prevent crashes, which can improve traffic safety.

Toward a Theory of Visual Information Acquisition in Driving

OBJECTIVE

The aim of this study is to describe information acquisition theory, explaining how drivers acquire and represent the information they need.

BACKGROUND

While questions of what drivers are aware of underlie many questions in driver behavior, existing theories do not directly address how drivers in particular and observers in general acquire visual information. Understanding the mechanisms of information acquisition is necessary to build predictive models of drivers' representation of the world and can be applied beyond driving to a wide variety of visual tasks.

METHOD

We describe our theory of information acquisition, looking to questions in driver behavior and results from vision science research that speak to its constituent elements. We focus on the intersection of peripheral vision, visual attention, and eye movement planning and identify how an understanding of these visual mechanisms and processes in the context of information acquisition can inform more complete models of driver knowledge and state.

RESULTS

We set forth our theory of information acquisition, describing the gap in understanding that it fills and how existing questions in this space can be better understood using it.

CONCLUSION

Information acquisition theory provides a new and powerful way to study, model, and predict what drivers know about the world, reflecting our current understanding of visual mechanisms and enabling new theories, models, and applications.

APPLICATION

Using information acquisition theory to understand how drivers acquire, lose, and update their representation of the environment will aid development of driver assistance systems, semiautonomous vehicles, and road safety overall.

The Influence of Visual-Manual Distractions on Anticipatory Driving

OBJECTIVE

The aim of this study is to investigate how anticipatory driving is influenced by distraction.

BACKGROUND

The anticipation of future events in traffic can allow potential gains in recognition and response times. Anticipatory actions (i.e., control actions in preparation for potential traffic changes) have been found to be more prevalent among experienced drivers in simulator studies when driving was the sole task. Despite the prevalence of visual-manual distractions and their negative effects on road safety, their influence on anticipatory driving has not yet been investigated beyond hazard anticipation.

METHODS

A simulator experiment was conducted with 16 experienced and 16 novice drivers. Half of the participants were provided with a self-paced visual-manual secondary task presented on a dashboard display.

RESULTS

More anticipatory actions were observed among experienced drivers; experienced drivers also exhibited more efficient visual scanning behaviors as indicated by higher glance rates toward and percent times looking at cues that facilitate the anticipation of upcoming events. Regardless of experience, those with the secondary task displayed reduced anticipatory actions and paid less attention toward anticipatory cues. However, experienced drivers had lower odds of exhibiting long glances toward the secondary task compared to novices. Further, the inclusion of glance duration on anticipatory cues increased the accuracy of a model predicting anticipatory actions based on on-road glance durations.

CONCLUSION

The results provide additional evidence to existing literature supporting the role of driving experience and distraction engagement in anticipatory driving.

APPLICATION

These findings can guide the design of in-vehicle systems and guide training programs to support anticipatory driving.

European NCAP Program Developments to Address Driver Distraction, Drowsiness and Sudden Sickness

Driver distraction and drowsiness remain significant contributors to death and serious injury on our roads and are long standing issues in road safety strategies around the world. With developments in automotive technology, including driver monitoring, there are now more options available for automotive manufactures to mitigate risks associated with driver state. Such developments in Occupant Status Monitoring (OSM) are being incorporated into the European New Car Assessment Programme (Euro NCAP) Safety Assist protocols. The requirements for OSM technologies are discussed along two dimensions: detection difficulty and behavioral complexity. More capable solutions will be able to provide higher levels of system availability, being the proportion of time a system could provide protection to the driver, and will be able to capture a greater proportion of complex real-word driver behavior. The testing approach could initially propose testing using both a dossier of evidence provided by the Original Equipment Manufacturer (OEM) alongside selected use of track testing. More capable systems will not rely only on warning strategies but will also include intervention strategies when a driver is not attentive. The roadmap for future OSM protocol development could consider a range of known and emerging safety risks including driving while intoxicated by alcohol or drugs, cognitive distraction, and the driver engagement requirements for supervision and take-over performance with assisted and automated driving features.

Keeping the driver in the loop through semi-automated or manual lane changes in conditionally automated driving

In the current study we investigated if drivers of conditionally automated vehicles can be kept in the loop through lane change maneuvers. More specifically, we examined whether involving drivers in lane-changes during a conditionally automated ride can influence critical take-over behavior and keep drivers' gaze on the road. In a repeated measures driving simulator study (n = 85), drivers drove the same route three times, each trial containing four lane changes that were all either (1) automated, (2) semi-automated or (3) manual. Each ride ended with a critical take-over situation that could be solved by braking and/or steering. Critical take-over reactions were analyzed with a linear mixed model and parametric accelerated failure time survival analysis. As expected, semi-automated and manual lane changes throughout the ride led to 13.5% and 17.0% faster maximum deceleration compared to automated lane changes. Additionally, semi-automated and manual lane changes improved the quality of the take-over by significantly decreasing standard deviation of the steering wheel angle. Unexpectedly, drivers in the semi-automated condition were slowest to start the braking maneuver. This may have been caused by the drivers' confusion as to how the semi-automated system would react. Additionally, the percentage gaze off-the-road was significantly decreased by the semi-automated (6.0%) and manual (6.6%) lane changes. Taken together, the results suggest that semi-automated and manual transitions may be an alarm-free instrument which developers could use to help maintain drivers' perception-action loop and improve automated driving safety.

Patterns in transitions of visual attention during baseline driving and during interaction with visual-manual and voice-based interfaces

Voice interfaces reduce visual demand compared with visual-manual interfaces, but the extent depends on design. This study compared visual demand during baseline driving with driving while using voice or manual inputs to place calls with Chevrolet MyLink, Volvo Sensus, or a smartphone. Mean glance duration and total eyes-off-road-time increased when using manual input compared with baseline driving; only eyes off road time increased with voice input. Confusion matrices developed with hidden Markov modelling characterise the similarity of glance sequences during baseline driving and while making phone calls. Glance sequences with the MyLink voice interface were misclassified as baseline driving more frequently than the other voice interfaces. Conversely, glance sequences with the Sensus and smartphone voice interfaces were more often misclassified as manual phone calling. Thus, the MyLink voice interface not only reduced the overall visual demand of placing calls, but produced glance patterns more similar to driving without another task. Practitioner Summary: The attention map and confusion matrix methodologies provide ways of characterising similarities and differences in glance behaviour across secondary task conditions, complementing traditional temporally based metrics (e.g. mean glance duration, long duration glances) while addressing some of the limitations of total-eyes-off-road-time (TEORT) for comparing secondary task behaviour to baseline driving.

A model for naturalistic glance behavior around Tesla Autopilot disengagements

OBJECTIVE

We present a model for visual behavior that can simulate the glance pattern observed around driver-initiated, non-critical disengagements of Tesla's Autopilot (AP) in naturalistic highway driving.

BACKGROUND

Drivers may become inattentive when using partially-automated driving systems. The safety effects associated with inattention are unknown until we have a quantitative reference on how visual behavior changes with automation.

METHODS

The model is based on glance data from 290 human initiated AP disengagement epochs. Glance duration and transition were modelled with Bayesian Generalized Linear Mixed models.

RESULTS

The model replicates the observed glance pattern across drivers. The model's components show that off-road glances were longer with AP active than without and that their frequency characteristics changed. Driving-related off-road glances were less frequent with AP active than in manual driving, while non-driving related glances to the down/center-stack areas were the most frequent and the longest (22% of the glances exceeded 2 s). Little difference was found in on-road glance duration.

CONCLUSION

Visual behavior patterns change before and after AP disengagement. Before disengagement, drivers looked less on road and focused more on non-driving related areas compared to after the transition to manual driving. The higher proportion of off-road glances before disengagement to manual driving were not compensated by longer glances ahead.

APPLICATION

The model can be used as a reference for safety assessment or to formulate design targets for driver management systems.

Determining the impact of augmented reality graphic spatial location and motion on driver behaviors

While researchers have explored benefits of adding augmented reality graphics to vehicle displays, the impact of graphic characteristics have not been well researched. In this paper, we consider the impact of augmented reality graphic spatial location and motion, as well as turn direction, traffic presence, and gender, on participant driving and glance behavior and preferences. Twenty-two participants navigated through a simulated environment while using four different graphics. We employed a novel glance allocation analysis to differentiate information likely gathered with each glace with more granularity. Fixed graphics generally resulted in less visual attention and more time scanning for hazards than animated graphics. Finally, the screen-fixed graphic was preferred by participants over all world-relative graphics, suggesting that graphic spatially integration into the world may not always be necessary in visually complex urban environments like those considered in this study.

Does agreement mean accuracy? Evaluating glance annotation in naturalistic driving data

Naturalistic driving studies often make use of cameras to monitor driver behavior. To analyze the resulting video images, human annotation is often adopted. These annotations then serve as the ‘gold standard’ to train and evaluate automated computer vision algorithms, even though it is uncertain how accurate human annotation is. In this study, we provide a first evaluation of glance direction annotation by comparing instructed, actual glance direction of truck drivers with annotated direction. Findings indicate that while for some locations high annotation accuracy is achieved, for most locations accuracy is well below 50%. Higher accuracy can be obtained by clustering these locations, but this also leads to reduced detail of the annotation, suggesting that decisions to use clustering should take into account the purpose of the annotation. The data also show that high agreement between annotators does not guarantee high accuracy. We argue that the accuracy of annotation needs to be verified experimentally more often.

In-vehicle displays to support driver anticipation of traffic conflicts in automated vehicles

OBJECTIVE

This paper investigates the effectiveness of in-vehicle displays in supporting drivers' anticipation of traffic conflicts in automated vehicles (AVs).

BACKGROUND

Providing takeover requests (TORs) along with information on automation capability (AC) has been found effective in supporting AV drivers' reactions to traffic conflicts. However, it is unclear what type of information can support drivers in anticipating traffic conflicts, so they can intervene (pre-event action) or prepare to intervene (pre-event preparation) proactively to avert them.

METHOD

In a driving simulator study with 24 experienced and 24 novice drivers, we evaluated the effectiveness of two in-vehicle displays in supporting anticipatory driving in AVs with adaptive cruise control and lane keeping assistance: TORAC (TOR + AC information) and STTORAC displays (surrounding traffic (ST) information + TOR + AC information). Both displays were evaluated against a baseline display that only showed whether the automation was engaged.

RESULTS

Compared to the baseline display, STTORAC led to more anticipatory driving behaviors (pre-event action or pre-event preparation) while TORAC led to less, along with decreased attention to environmental cues that indicated an upcoming event. STTORAC led to the highest level of driving safety, as indicated by minimum gap time for scenarios that required driver intervention, followed by TORAC, and then the baseline display.

CONCLUSIONS

Providing surrounding traffic information to drivers of AVs, in addition to TORs and automation capability information, can support their anticipation of potential traffic conflicts. Without the surrounding traffic information, drivers can over-rely on displays that provide TORs and automation capability information.

Redesigning Today's Driving Automation Toward Adaptive Backup Control With Context-Based and Invisible Interfaces

OBJECTIVE

We investigated a driver monitoring system (DMS) designed to adaptively back up distracted drivers with automated driving.

BACKGROUND

Humans are likely inadequate for supervising today's on-road driving automation. Conversely, backup concepts can use eye-tracker DMS to retain the human as the primary driver and use computerized control only if needed. A distraction DMS where perceived false alarms are minimized and the status of the backup is unannounced might reduce problems of distrust and overreliance, respectively. Experimental research is needed to assess the viability of such designs.

METHODS

In a driving simulator, 91 participants either supervised driving automation (auto-hand-on-wheel vs. auto-hands-off-wheel), drove with different forms of DMS-induced backup control (eyes-only-backup vs. eyes-plus-context-backup; visible-backup vs. invisible-backup), or drove without any automation. All participants performed a visual N-back task throughout.

RESULTS

Supervised driving automation increased visual distraction and hazard non-responses compared to backup and conventional driving. Auto-hand-on-wheel improved response generation compared to auto-hands-off-wheel. Across entire driving trials, the backup improved lateral performance compared to conventional driving. Without negatively impacting safety, the eyes-plus-context-backup DMS reduced unnecessary automated control compared to the eyes-only-backup DMS conditions. Eyes-only-backup produced low satisfaction ratings, whereas eyes-plus-context-backup satisfaction was on par with automated driving. There were no appreciable negative consequences attributable to the invisible-backup driving automation.

CONCLUSIONS

We have demonstrated preliminary feasibility of DMS designs that incorporate driving context information for distraction assessment and suppress their status indication.

APPLICATION

An appropriately designed DMS can enable benefits for automated driving as a backup.

AR DriveSim: An Immersive Driving Simulator for Augmented Reality Head-Up Display Research

Optical see-through automotive head-up displays (HUDs) are a form of augmented reality (AR) that is quickly gaining penetration into the consumer market. Despite increasing adoption, demand, and competition among manufacturers to deliver higher quality HUDs with increased fields of view, little work has been done to understand how best to design and assess AR HUD user interfaces, and how to quantify their effects on driver behavior, performance, and ultimately safety. This paper reports on a novel, low-cost, immersive driving simulator created using a myriad of custom hardware and software technologies specifically to examine basic and applied research questions related to AR HUDs usage when driving. We describe our experiences developing simulator hardware and software and detail a user study that examines driver performance, visual attention, and preferences using two AR navigation interfaces. Results suggest that conformal AR graphics may not be inherently better than other HUD interfaces. We include lessons learned from our simulator development experiences, results of the user study and conclude with limitations and future work.

Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

Operators of highly automated driving systems may exhibit behaviour characteristic for overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. A driving simulator study was used to assess the impact of dynamically communicating system uncertainties on monitoring, trust, workload, takeovers, and physiological responses. The uncertainty information was conveyed visually using a stylised heart beat combined with a numerical display and users were engaged in a visual search task. Multilevel analysis results suggest that uncertainty communication helps operators calibrate their trust and gain situation awareness prior to critical situations, resulting in safer takeovers. In addition, eye tracking data indicate that operators can adjust their gaze behaviour in correspondence with the level of uncertainty. However, conveying uncertainties using a visual display significantly increases operator workload and impedes users in the execution of non-driving related tasks. Practitioner Summary: This article illustrates how the communication of system uncertainty information helps operators calibrate their trust in automation and, consequently, gain situation awareness. Multilevel analysis results of a driving simulator study affirm the benefits for trust calibration and highlight that operators adjust their behaviour according to multiple uncertainty levels.

The role of peripheral vision in vertical road sign identification and discrimination

The role of peripheral vision in road sign identification and discrimination was investigated in two studies. Peripheral vision plays an important role in road signs perception due to their lateral positioning. In the first study 20 participants identified road signs presented at five levels of horizontal eccentricity (1.1°-12.4°), and two levels of vertical eccentricity (0°-2.5°). In the second study road sign discrimination was tested in a same-different discrimination task. The first study showed that a vertical offset of 2.5° degraded proportion correct rate by 9%. Proportion correct rate decreased from 79% to 41% in the transition from 1.1° to 12.4° of horizontal offset. The second study showed an accurate discrimination for road signs presented within a horizontal offset of 6.4°. Road signs with angular shapes and prominent vertexes as triangular or cross signs were better identified in peripheral vision than signs with more compact shapes (circular signs). Practitioner summary: Vertical road signs, due to their lateral positioning, are often perceived in peripheral vision. Horizontal and vertical eccentricity negatively impacts the driver's ability to correctly identify and discriminate traffic signs. The use of singular shapes and a design with simple pictograms and large contrasting areas strongly facilitate road sign perception in peripheral vision.