A Systematic Review and Meta-Analysis of Takeover Performance During Conditionally Automated Driving

Exploring the Timing of Disengagement From Nondriving Related Tasks in Scheduled Takeovers With Pre-Alerts: An Analysis of Takeover-Related Measures

OBJECTIVES

This study aimed to investigate drivers' disengagement from nondriving related tasks (NDRT) during scheduled takeovers and to evaluate its impact on takeover performance.

BACKGROUND

During scheduled takeovers, drivers typically have sufficient time to prepare. However, inadequate disengagement from NDRTs can introduce safety risks.

METHOD

Participants experienced scheduled takeovers using a driving simulator, undergoing two conditions, with and without an NDRT. We assessed their takeover performance and monitored their NDRT disengagement from visual, cognitive, and physical perspectives.

RESULTS

The study examined three NDRT disengagement timings (DTs): DT1 (disengaged before the takeover request), DT2 (disengaged after the request but before taking over), and DT3 (not disengaged). The impact of NDRT on takeover performance varied depending on DTs. Specifically, DT1 demonstrated no adverse effects; DT2 impaired takeover time, while DT3 impaired both takeover time and quality. Additionally, participants who displayed DT1 exhibited longer eye-off-NDRT duration and a higher eye-off-NDRT count during the prewarning stage compared to those with DT2 and DT3.

CONCLUSION

Drivers can benefit from earlier disengagement from NDRTs, demonstrating resilience to the adverse effects of NDRTs on takeover performance. The disengagement of cognition is often delayed compared to that of eyes and hands, potentially leading to DT3. Moreover, visual disengagement from NDRTs during the prewarning stage could distinguish DT1 from the other two.

APPLICATION

Our study emphasizes considering NDRT disengagement in designing systems for scheduled takeovers. Measures should be taken to promote early disengagement, facilitate cognitive disengagement, and employ visual disengagement during the prewarning period as predictive indicators of DTs.

Post Take-Over Performance Varies in Drivers of Automated and Connected Vehicle Technology in Near-Miss Scenarios

OBJECTIVE

This study examined the impact of monitoring instructions when using an automated driving system (ADS) and road obstructions on post take-over performance in near-miss scenarios.

BACKGROUND

Past research indicates partial ADS reduces the driver's situation awareness and degrades post take-over performance. Connected vehicle technology may alert drivers to impending hazards in time to safely avoid near-miss events.

METHOD

Forty-eight licensed drivers using ADS were randomly assigned to either the active driving or passive driving condition. Participants navigated eight scenarios with or without a visual obstruction in a distributed driving simulator. The experimenter drove the other simulated vehicle to manually cause near-miss events. Participants' mean longitudinal velocity, standard deviation of longitudinal velocity, and mean longitudinal acceleration were measured.

RESULTS

Participants in passive ADS group showed greater, and more variable, deceleration rates than those in the active ADS group. Despite a reliable audiovisual warning, participants failed to slow down in the red-light running scenario when the conflict vehicle was occluded. Participant's trust in the automated driving system did not vary between the beginning and end of the experiment.

CONCLUSION

Drivers interacting with ADS in a passive manner may continue to show increased and more variable deceleration rates in near-miss scenarios even with reliable connected vehicle technology. Future research may focus on interactive effects of automated and connected driving technologies on drivers' ability to anticipate and safely navigate near-miss scenarios.

APPLICATION

Designers of automated and connected vehicle technologies may consider different timing and types of cues to inform the drivers of imminent hazard in high-risk scenarios for near-miss events.

Effect of Takeover Request Time and Warning Modality on Trust in L3 Automated Driving

OBJECTIVE

This study investigated the effects of four takeover request (TOR) times and seven warning modalities on performance and trust in automated driving on a mildly congested urban road scenario, as well as the relationship between takeover performance and trust.

BACKGROUND

Takeover is crucial in L3 automated driving, where human-machine codriving is employed. Establishing trust in takeover scenarios among drivers can enhance the acceptance of autonomous vehicles, thereby promoting their widespread adoption.

METHOD

Using a driving simulator, data from 28 participants, including collision counts, takeover time (ToT), electrodermal activity (EDA) data, and self-reported trust scores, were collected and analyzed primarily using Generalized Linear Mixed Models (GLMM).

RESULTS

Collisions during the takeover undermined participants' trust in the autonomous driving system. As TOR time increased, participants' trust improved, and the longer TOR time did not lead to participant confusion. There was no significant relationship between warning modality and trust. Furthermore, the combination of three warning modalities did not exhibit a notable advantage over the combination of two modalities.

CONCLUSION

The study examined the effects of TOR time and warning modality on trust, as well as preliminarily explored the potential association between takeover performance, including collisions and ToT, and trust in autonomous driving takeovers.

APPLICATION

Researchers and designers of automotive interactions were given referenceable TOR time and warning modality by this study, which extended the autonomous driving takeover scenarios. These findings contributed to boosting drivers' confidence in transferring control to the automated system.

Driver Situation Awareness for Regaining Control from Conditionally Automated Vehicles: A Systematic Review of Empirical Studies

OBJECTIVE

An up-to-date and thorough literature review is needed to identify factors that influence driver situation awareness (SA) during control transitions in conditionally automated vehicles (AV). This review also aims to ascertain SA components required for takeovers, aiding in the design and evaluation of human-vehicle interfaces (HVIs) and the selection of SA assessment methodologies.

BACKGROUND

Conditionally AVs alleviate the need for continuous road monitoring by drivers yet necessitate their reengagement during control transitions. In these instances, driver SA is crucial for effective takeover decisions and subsequent actions. A comprehensive review of influential SA factors, SA components, and SA assessment methods will facilitate driving safety in conditionally AVs but is still lacking.

METHOD

A systematic literature review was conducted. Thirty-four empirical research articles were screened out to meet the criteria for inclusion and exclusion.

RESULTS

A conceptual framework was developed, categorizing 23 influential SA factors into four clusters: task/system, situational, individual, and nondriving-related task factors. The analysis also encompasses an examination of pertinent SA components and corresponding HVI designs for specific takeover events, alongside an overview of SA assessment methods for conditionally AV takeovers.

CONCLUSION

The development of a conceptual framework outlining influential SA factors, the examination of SA components and their suitable design of presentation, and the review of SA assessment methods collectively contribute to enhancing driving safety in conditionally AVs.

APPLICATION

This review serves as a valuable resource, equipping researchers and practitioners with insights to guide their efforts in evaluating and enhancing driver SA during conditionally AV takeovers.

Predicting driver's takeover time based on individual characteristics, external environment, and situation awareness

The driver's takeover time is crucial to ensure a safe takeover transition in conditional automated driving. The study aimed to construct a prediction model of driver's takeover time based on individual characteristics, external environment, and situation awareness variables. A total of 18 takeover events were designed with scenarios, non-driving-related tasks, takeover request time, and traffic flow as variables. High-fidelity driving simulation experiments were carried out, through which the driver's takeover data was obtained. Fifteen basic factors and three dynamic factors were extracted from individual characteristics, external environment, and situation awareness. In this experiment, these 18 factors were selected as input variables, and XGBoost and Shapely were used as prediction methods. A takeover time prediction model (BM + SA model) was then constructed. Moreover, we analyzed the main effect of input variables on takeover time, and the interactive contribution made by the variables. And in this experiment, the 15 basic factors were selected as input variables, and the basic takeover time prediction model (BM model) was constructed. In addition, this study compared the performance of the two models and analyzed the contribution of input variables to takeover time. The results showed that the goodness of fit of the BM + SA model (Adjusted_R2) was 0.7746. The XGBoost model performs better than other models (support vector machine, random forest, CatBoost, and LightBoost models). The relative importance degree of situation awareness variables, individual characteristic variables, and external environment variables to takeover time gradually reduced. Takeover time increased with the scan and gaze durations and decreased with pupil area and self-reported situation awareness scores. There was also an interaction effect between the variables to affect takeover time. Overall, the performance of the BM + SA model was better than that of the BM model. This study can provide support for predicting driver's takeover time and analyzing the mechanism of influence on takeover time. This study can provide support for the development of real-time driver's takeover ability prediction systems and optimization of human-machine interaction design in automated vehicles, as well as for the management department to evaluate and improve the driver's takeover performance in a targeted manner.

Effects of driving style on takeover performance during automated driving: Under the influence of warning system factors

Driving style has been proposed to be a critical factor in automated driving. However, the role of driving style in the process of taking over during automated driving needs further investigation. The main purpose of this study was to investigate the influence of driving style on takeover performance under the influence of warning system factors. In addition, this study also explored whether the impact of driving style on reaction time varies over time and the role of driving style on a comprehensive takeover quality indicator. Two driving simulation experiments with different takeover request (TOR) designs were conducted. In experiment 1, content warning information was provided in the TOR with different warning stage designs; in experiment 2, countdown warning information was provided in the TOR with different warning stage designs. Sixty-four participants (32 for experiment 1 and 32 for experiment 2) were classified into two groups based on their driving style (i.e., aggressive, or defensive) using the Chinese version of the Multidimensional Driving Style Inventory (the brief MDSI-C). The results suggested that drivers' driving style had significant effects on takeover performance, but the effects were influenced by warning system designs. Specifically, defensive participants performed better takeover performance, i.e., shorter reaction time and cautious vehicle control behaviors, than aggressive participants in most warning conditions. The content and countdown warning information and warning stage design affected the roles of driving style on takeover performance: 1) compared to the one-stage warning design, the two-stage warning design significantly shortened the reaction time of the participants with different driving styles, 2) compared to the countdown warning information design, the design of content warning information can shorten the reaction time of aggressive participants and lengthen the reaction time of defensive participants in the two-stage warning conditions, and 3) compared to the content warning information design, countdown warning information can improve the safe takeover performance of defensive participants. This study provides a better understanding of the role of driving style on takeover performance, and driving style should be considered when designing warning systems for autonomous vehicles.

Agent Transparency, Situation Awareness, Mental Workload, and Operator Performance: A Systematic Literature Review

OBJECTIVE

In this review, we investigate the relationship between agent transparency, Situation Awareness, mental workload, and operator performance for safety critical domains.

BACKGROUND

The advancement of highly sophisticated automation across safety critical domains poses a challenge for effective human oversight. Automation transparency is a design principle that could support humans by making the automation's inner workings observable (i.e., "seeing-into"). However, experimental support for this has not been systematically documented to date.

METHOD

Based on the PRISMA method, a broad and systematic search of the literature was performed focusing on identifying empirical research investigating the effect of transparency on central Human Factors variables.

RESULTS

Our final sample consisted of 17 experimental studies that investigated transparency in a controlled setting. The studies typically employed three human-automation interaction types: responding to agent-generated proposals, supervisory control of agents, and monitoring only. There is an overall trend in the data pointing towards a beneficial effect of transparency. However, the data reveals variations in Situation Awareness, mental workload, and operator performance for specific tasks, agent-types, and level of integration of transparency information in primary task displays.

CONCLUSION

Our data suggests a promising effect of automation transparency on Situation Awareness and operator performance, without the cost of added mental workload, for instances where humans respond to agent-generated proposals and where humans have a supervisory role.

APPLICATION

Strategies to improve human performance when interacting with intelligent agents should focus on allowing humans to see into its information processing stages, considering the integration of information in existing Human Machine Interface solutions.

Drivers' Performance in Non-critical Take-Overs From an Automated Driving System-An On-Road Study

OBJECTIVE

The objective of this semi-controlled study was to investigate drivers' performance when resuming control from an Automated Driving System (ADS), simulated through the Wizard of Oz method, in real traffic.

BACKGROUND

Research on take-overs has primarily focused on urgent scenarios. This article aims to shift the focus to non-critical take-overs from a system operating in congested traffic situations.

METHOD

Twenty drivers drove a selected route in rush-hour traffic in the San Francisco Bay Area, CA, USA. During the drive, the ADS became available when predetermined availability conditions were fulfilled. When the system was active, the drivers were free to engage in non-driving related activities.

RESULTS

The results show that drivers' transition time goes down with exposure, making it reasonable to assume that some experience is required to regain control with comfort and ease. The novel analysis of after-effects of automated driving on manual driving performance implies that the after-effects were close to negligible. Observational data indicate that, with exposure, a majority of the participants started to engage in non-driving related activities to some extent, but it is unclear how the activities influenced the take-over performance.

CONCLUSION

The results indicate that drivers need repeated exposure to take-overs to be able to fully resume manual control with ease.

APPLICATION

Take-over signals (e.g., visuals, sounds, and haptics) should be carefully designed to avoid startle effects and the human-machine interface should provide clear guidance on the required take-over actions.

Is driving experience all that matters? Drivers' takeover performance in conditionally automated driving

INTRODUCTION

In conditionally automated driving, drivers are allowed to engage in non-driving related tasks (NDRTs) and are occasionally requested to take over vehicle control in situations that the automation system cannot handle. Drivers may not be able to adequately perform such requests if they have limited driving experience. This study investigates the influence of driving experience on takeover performance in conditionally automated driving.

METHOD

Nineteen subjects participated in this driving simulator study. The NDRTs consisted of three tasks: writing business emails (working condition), watching videos (entertaining condition), and taking a break with eyes closed (resting condition). These three NDRTs require drivers to invest high, moderate, and low levels of mental workload, respectively. The duration of engagement in each NDRT before a takeover request (TOR) was either 5 minutes (short interval) or 30 minutes (long interval).

RESULTS

Drivers' driving experience and performance during the control period are highly correlated with their TOR performance. Furthermore, the type and duration of NDRT influence TOR performance, and inexperienced drivers exhibit poorer TOR performance than experienced drivers.

CONCLUSIONS AND PRACTICAL APPLICATIONS

These findings have relevance for the types of NDRTs that ought to be permitted during automated driving, the design of automated driving systems, and the formulation of regulations regarding the responsible use of automated vehicles.

Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving

The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.

Assessing the physiological effect of non-driving-related task performance and task modality in conditionally automated driving systems: A systematic review and meta-analysis

In conditionally automated driving, the driver is free to disengage from controlling the vehicle, but they are expected to resume driving in response to certain situations or events that the system is not equipped to respond to. As the level of vehicle automation increases, drivers often engage in non-driving-related tasks (NDRTs), defined as any secondary task unrelated to the primary task of driving. This engagement can have a detrimental effect on the driver's situation awareness and attentional resources. NDRTs with resource demands that overlap with the driving task, such as visual or manual tasks, may be particularly deleterious. Therefore, monitoring the driver's state is an important safety feature for conditionally automated vehicles, and physiological measures constitute a promising means of doing this. The present systematic review and meta-analysis synthesises findings from 32 studies concerning the effect of NDRTs on drivers' physiological responses, in addition to the effect of NDRTs with a visual or a manual modality. Evidence was found that NDRT engagement led to higher physiological arousal, indicated by increased heart rate, electrodermal activity and a decrease in heart rate variability. There was mixed evidence for an effect of both visual and manual NDRT modalities on all physiological measures. Understanding the relationship between task performance and arousal during automated driving is of critical importance to the development of driver monitoring systems and improving the safety of this technology.

Assessing the physiological effect of non-driving-related task performance in conditionally automated driving systems: A systematic review and meta-analysis protocol

Background

Level 3 automated driving systems involve the continuous performance of the driving task by artificial intelligence within set environmental conditions, such as a straight highway. The driver's role in Level 3 is to resume responsibility of the driving task in response to any departure from these conditions. As automation increases, a driver's attention may divert towards non-driving-related tasks (NDRTs), making transitions of control between the system and user more challenging. Safety features such as physiological monitoring thus become important with increasing vehicle automation. However, to date there has been no attempt to synthesise the evidence for the effect of NDRT engagement on drivers' physiological responses in Level 3 automation.

Methods

A comprehensive search of the electronic databases MEDLINE, EMBASE, Web of Science, PsycINFO, and IEEE Explore will be conducted. Empirical studies assessing the effect of NDRT engagement on at least one physiological parameter during Level 3 automation, in comparison with a control group or baseline condition will be included. Screening will take place in two stages, and the process will be outlined within a PRISMA flow diagram. Relevant physiological data will be extracted from studies and analysed using a series of meta-analyses by outcome. A risk of bias assessment will also be completed on the sample.

Conclusion

This review will be the first to appraise the evidence for the physiological effect of NDRT engagement during Level 3 automation, and will have implications for future empirical research and the development of driver state monitoring systems.

Examining the Effects of Visibility and Time Headway on the Takeover Risk during Conditionally Automated Driving

The objective of this study is to examine the effects of visibility and time headway on the takeover performance in L3 automated driving. Both non-critical and critical driving scenarios were considered by changing the acceleration value of the leading vehicle. A driving simulator experiment with 18 driving scenarios was conducted and 30 participants complete the experiment. Based on the data obtained from the experiment, the takeover reaction time, takeover control time, and takeover responses were analyzed. The minimum Time-To-Collision (Min TTC) was used to measure the takeover risk level and a binary logit model for takeover risk levels was estimated. The results indicate that the visibility distance (VD) has no significant effects on the takeover control time, while the time headway (THW) and the acceleration of the leading vehicle (ALV) could affect the takeover control time significantly; most of the participants would push the gas pedal to accelerate the ego vehicle as the takeover response under non-critical scenarios, while braking was the dominant takeover response for participants in critical driving scenarios; decreasing the TCT and taking the appropriate takeover response would reduce the takeover risk significantly, so it is suggested that the automation system should provide the driver with the urgency of the situation ahead and the tips for takeover responses by audio prompts or the head-up display. This study is expected to facilitate the overall understanding of the effects of visibility and time headway on the takeover performance in conditionally automated driving.

Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.