A Review of Non-driving-related Tasks Used in Studies on Automated Driving

Assessing the physiological effect of non-driving-related task performance and task modality in conditionally automated driving systems: A systematic review and meta-analysis

In conditionally automated driving, the driver is free to disengage from controlling the vehicle, but they are expected to resume driving in response to certain situations or events that the system is not equipped to respond to. As the level of vehicle automation increases, drivers often engage in non-driving-related tasks (NDRTs), defined as any secondary task unrelated to the primary task of driving. This engagement can have a detrimental effect on the driver's situation awareness and attentional resources. NDRTs with resource demands that overlap with the driving task, such as visual or manual tasks, may be particularly deleterious. Therefore, monitoring the driver's state is an important safety feature for conditionally automated vehicles, and physiological measures constitute a promising means of doing this. The present systematic review and meta-analysis synthesises findings from 32 studies concerning the effect of NDRTs on drivers' physiological responses, in addition to the effect of NDRTs with a visual or a manual modality. Evidence was found that NDRT engagement led to higher physiological arousal, indicated by increased heart rate, electrodermal activity and a decrease in heart rate variability. There was mixed evidence for an effect of both visual and manual NDRT modalities on all physiological measures. Understanding the relationship between task performance and arousal during automated driving is of critical importance to the development of driver monitoring systems and improving the safety of this technology.

Repeated conditionally automated driving on the road: How do drivers leave the loop over time?

The goal of this on the road driving study was to investigate how drivers adapt their behavior when driving with conditional vehicle automation (SAE L3) on different occasions. Specifically, we focused on changes in how fast drivers took over control from automation and how their gaze off the road changed over time. On each of three consecutive days, 21 participants drove for 50 min, in a conditionally automated vehicle (Wizard of Oz methodology), on a typical German commuting highway. Over these rides the take-over behavior and gaze behavior were analyzed. The data show that drivers' reactions to non-critical, system initiated, take-overs took about 5.62 s and did not change within individual rides, but on average became 0.72 s faster over the three rides. After these self-paced take-over requests a final urgent take-over request was issued at the end of the third ride. In this scenario participants took over rapidly with an average of 5.28 s. This urgent take-over time was not found to be different from the self-paced take-over requests in the same ride. Regarding gaze behavior, participants' overall longest glance off the road and the percentage of time looked off the road increased within each ride, but stayed stable over the three rides. Taken together, our results suggest that drivers regularly leave the loop by gazing off the road, but multiple exposures to take-over situations in automated driving allow drivers to come back into loop faster.

Assessing the physiological effect of non-driving-related task performance in conditionally automated driving systems: A systematic review and meta-analysis protocol

Background

Level 3 automated driving systems involve the continuous performance of the driving task by artificial intelligence within set environmental conditions, such as a straight highway. The driver's role in Level 3 is to resume responsibility of the driving task in response to any departure from these conditions. As automation increases, a driver's attention may divert towards non-driving-related tasks (NDRTs), making transitions of control between the system and user more challenging. Safety features such as physiological monitoring thus become important with increasing vehicle automation. However, to date there has been no attempt to synthesise the evidence for the effect of NDRT engagement on drivers' physiological responses in Level 3 automation.

Methods

A comprehensive search of the electronic databases MEDLINE, EMBASE, Web of Science, PsycINFO, and IEEE Explore will be conducted. Empirical studies assessing the effect of NDRT engagement on at least one physiological parameter during Level 3 automation, in comparison with a control group or baseline condition will be included. Screening will take place in two stages, and the process will be outlined within a PRISMA flow diagram. Relevant physiological data will be extracted from studies and analysed using a series of meta-analyses by outcome. A risk of bias assessment will also be completed on the sample.

Conclusion

This review will be the first to appraise the evidence for the physiological effect of NDRT engagement during Level 3 automation, and will have implications for future empirical research and the development of driver state monitoring systems.

Takeover requests for automated driving: The effects of signal direction, lead time, and modality on takeover performance

Vehicle-to-driver takeover will still be needed in semi-autonomous vehicles. Due to the complexity of the takeover process, it is important to develop interfaces to support good takeover performance. Multimodal displays have been proposed as a candidate for the design of takeover requests (TORs), but many questions remain unanswered regarding the effectiveness of this approach. This study investigated the effects of takeover signal direction (ipsilateral vs. contralateral), lead time (4 vs. 7 s), and modality (uni-, bi-, and trimodal combinations of visual, auditory, and tactile signals) on automated vehicle takeover performance. Twenty-four participants rode in a simulated SAE Level 3 vehicle and performed a series of takeover tasks when presented with a TOR. Overall, single and multimodal signals with a tactile component were correlated with the faster takeover and information processing times, and were perceived as most useful. Ipsilateral signals showed a marginally significant benefit to takeover times compared to contralateral signals. Finally, a shorter lead time was associated with faster takeover times, but also poorer takeover quality. Findings from this study can inform the design of in-vehicle information and warning systems for next-generation transportation.

Mental Fatigue and Basketball Performance: A Systematic Review

Mental fatigue (MF) is a psycho-biological state that impairs sports-related performances. Recently, it has been proved that MF can affect basketball performance. However, a systematic overview detailing the influences of MF on basketball performance is still lacking. This study aims to investigate the effects of MF on the physical, technical, tactical, and cognitive performance of basketball. We used the databases of PubMed, Web of Science, SPORTDiscus, Scopes, and CKNI for articles published up to 31 May 2021. The articles included in this study were projected to test whether MF influences basketball athlete performance. Only experimental design studies were selected, and the control condition was without MF. Finally, seven articles fit the inclusion criteria. The results imply that MF impairs the technical aspects of basketball (free throws, three-point shots, and total turnover) and the players' cognitive [take-the-first (TTF) heuristics and decision-making] performance, which results in athletes not using their techniques skillfully and being unable to make practical decisions during critical points in the game. In addition to that, the influences of MF on physical and tactical performance have not been studied. Further studies should look into comprehensive research on the influences of MF on basketball performance, especially on a player's physical and tactical performance.

Systematic Review Registration: [https://inplasy.com/] [INPLASY2021100017].

Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.

Optimal Time Intervals in Two-Stage Takeover Warning Systems With Insight Into the Drivers' Neuroticism Personality

Conditional automated driving [level 3, Society of Automotive Engineers (SAE)] requires drivers to take over the vehicle when an automated system's failure occurs or is about to leave its operational design domain. Two-stage warning systems, which warn drivers in two steps, can be a promising method to guide drivers in preparing for the takeover. However, the proper time intervals of two-stage warning systems that allow drivers with different personalities to prepare for the takeover remain unclear. This study explored the optimal time intervals of two-stage warning systems with insights into the drivers' neuroticism personality. A total of 32 drivers were distributed into two groups according to their self-ratings in neuroticism (high vs. low). Each driver experienced takeover under the two-stage warning systems with four time intervals (i.e., 3, 5, 7, and 9 s). The takeover performance (i.e., hands-on-steering-wheel time, takeover time, and maximum resulting acceleration) and subjective opinions (i.e., appropriateness and usefulness) for time intervals and situation awareness (SA) were recorded. The results showed that drivers in the 5-s time interval had the best takeover preparation (fast hands-on steering wheel responses and sufficient SA). Furthermore, both the 5- and 7-s time intervals resulted in more rapid takeover reactions and were rated more appropriate and useful than the 3- and 9-s time intervals. In terms of personality, drivers with high neuroticism tended to take over immediately after receiving takeover messages, at the cost of SA deficiency. In contrast, drivers with low neuroticism responded safely by judging whether they gained enough SA. We concluded that the 5-s time interval was optimal for drivers in two-stage takeover warning systems. When considering personality, drivers with low neuroticism had no strict requirements for time intervals. However, the extended time intervals were favorable for drivers with high neuroticism in developing SA. The present findings have reference implications for designers and engineers to set the time intervals of two-stage warning systems according to the neuroticism personality of drivers.

Mode Awareness and Automated Driving—What Is It and How Can It Be Measured?

In SAE (Society of Automotive Engineers) Level 2, the driver has to monitor the traffic situation and system performance at all times, whereas the system assumes responsibility within a certain operational design domain in SAE Level 3. The different responsibility allocation in these automation modes requires the driver to always be aware of the currently active system and its limits to ensure a safe drive. For that reason, current research focuses on identifying factors that might promote mode awareness. There is, however, no gold standard for measuring mode awareness and different approaches are used to assess this highly complex construct. This circumstance complicates the comparability and validity of study results. We thus propose a measurement method that combines the knowledge and the behavior pillar of mode awareness. The latter is represented by the relational attention ratio in manual, Level 2 and Level 3 driving as well as the controllability of a system limit in Level 2. The knowledge aspect of mode awareness is operationalized by a questionnaire on the mental model for the automation systems after an initial instruction as well as an extensive enquiry following the driving sequence. Further assessments of system trust, engagement in non-driving related tasks and subjective mode awareness are proposed.

Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures

OBJECTIVE

This article provides a review of empirical studies of automated vehicle takeovers and driver modeling to identify influential factors and their impacts on takeover performance and suggest driver models that can capture them.

BACKGROUND

Significant safety issues remain in automated-to-manual transitions of vehicle control. Developing models and computer simulations of automated vehicle control transitions may help designers mitigate these issues, but only if accurate models are used. Selecting accurate models requires estimating the impact of factors that influence takeovers.

METHOD

Articles describing automated vehicle takeovers or driver modeling research were identified through a systematic approach. Inclusion criteria were used to identify relevant studies and models of braking, steering, and the complete takeover process for further review.

RESULTS

The reviewed studies on automated vehicle takeovers identified several factors that significantly influence takeover time and post-takeover control. Drivers were found to respond similarly between manual emergencies and automated takeovers, albeit with a delay. The findings suggest that existing braking and steering models for manual driving may be applicable to modeling automated vehicle takeovers.

CONCLUSION

Time budget, repeated exposure to takeovers, silent failures, and handheld secondary tasks significantly influence takeover time. These factors in addition to takeover request modality, driving environment, non-handheld secondary tasks, level of automation, trust, fatigue, and alcohol significantly impact post-takeover control. Models that capture these effects through evidence accumulation were identified as promising directions for future work.

APPLICATION

Stakeholders interested in driver behavior during automated vehicle takeovers may use this article to identify starting points for their work.

Noncritical State Transitions During Conditionally Automated Driving on German Freeways: Effects of Non-Driving Related Tasks on Takeover Time and Takeover Quality

OBJECTIVE

This study aimed at investigating the driver's takeover performance when switching from working on different non-driving related tasks (NDRTs) while driving with a conditionally automated driving function (SAE L3), which was simulated by a Wizard of Oz vehicle, to manual vehicle control under naturalistic driving conditions.

BACKGROUND

Conditionally automated driving systems, which are currently close to market introduction, require the user to stay fallback ready. As users will be allowed to engage in more complex NDRTs during the automated drive than when driving manually, the time needed to regain full manual control could likely be increased.

METHOD

Thirty-four users engaged in different everyday NDRTs while driving automatically with a Wizard of Oz vehicle. After approximately either 5 min or 15 min of automated driving, users were requested to take back vehicle control in noncritical situations. The test drive took place in everyday traffic on German freeways in the metropolitan area of Stuttgart.

RESULTS

Particularly tasks that required users to turn away from the central road scene or hold an object in their hands led to increased takeover times. Accordingly, increased variance in the driver's lane position was found shortly after the switch to manual control. However, the drivers rated the takeover situations to be mostly "harmless."

CONCLUSION

Drivers managed to regain control over the vehicle safely, but they needed more time to prepare for the manual takeover when the NDRTs caused motoric workload.

APPLICATION

The timings found in the study can be used to design comfortable and safe takeover concepts for automated vehicles.