Enhancing safety in conditionally automated driving: Can more takeover request visual information make a difference in hazard scenarios with varied hazard visibility?

Autonomous driving technology has the potential to significantly reduce the number of traffic accidents. However, before achieving full automation, drivers still need to take control of the vehicle in complex and diverse scenarios that the autonomous driving system cannot handle. Therefore, appropriate takeover request (TOR) designs are necessary to enhance takeover performance and driving safety. This study focuses on takeover tasks in hazard scenarios with varied hazard visibility, which can be categorized as overt hazards and covert hazards. Through ergonomic experiments, the impact of TOR interface visual information, including takeover warning, hazard direction, and time to collision, on takeover performance is investigated, and specific analyses are conducted using eye-tracking data. The following conclusions are drawn from the experiments: (1) The visibility of hazards significantly affects takeover performance. (2) Providing more TOR visual information in hazards with different visibility has varying effects on drivers' visual attention allocation but can improve takeover performance. (3) More TOR visual information helps reduce takeover workload and increase human-machine trust. Based on these findings, this paper proposes the following TOR visual interface design strategies: (1) In overt hazard scenarios, only takeover warning is necessary, as additional visual information may distract drivers' attention. (2) In covert hazard scenarios, the TOR visual interface should better assist drivers in understanding the current hazard situation by providing information on hazard direction and time to collision to enhance takeover performance.

The effects of various auditory takeover requests: A simulated driving study considering the modality of non-driving-related tasks

With the era of automated driving approaching, designing an effective auditory takeover request (TOR) is critical to ensure automated driving safety. The present study investigated the effects of speech-based (speech and spearcon) and non-speech-based (earcon and auditory icon) TORs on takeover performance and subjective preferences. The potential impact of the non-driving-related task (NDRT) modality on auditory TORs was considered. Thirty-two participants were recruited in the present study and assigned to two groups, with one group performing the visual N-back task and another performing the auditory N-back task during automated driving. They were required to complete four simulated driving blocks corresponding to four auditory TOR types. The earcon TOR was found to be the most suitable for alerting drivers to return to the control loop because of its advantageous takeover time, lane change time, and minimum time to collision. Although participants preferred the speech TOR, it led to relatively poor takeover performance. In addition, the auditory NDRT was found to have a detrimental impact on auditory TORs. When drivers were engaged in the auditory NDRT, the takeover time and lane change time advantages of earcon TORs no longer existed. These findings highlight the importance of considering the influence of auditory NDRTs when designing an auditory takeover interface. The present study also has some practical implications for researchers and designers when designing an auditory takeover system in automated vehicles.

The monitoring requests on young driver's fatigue and take-over performance in prolonged conditional automated driving

INTRODUCTION

L3 automated vehicles can perform all dynamic driving tasks unless a take-over occurs due to operational limits. This issue is potentially important for young drivers who are vulnerable road users since they have skill deficits and easily evolve into aberrant driving. However, drivers lacking active involvement may be fatigued and drowsy. Previous research indicated that performing a voluntary non-driving-related task (NDRT) could keep drivers alert, but there was no difference in take-over performance with or without NDRT. Providing a monitoring request (MR) before a possible take-over request (TOR) exhibited better take-over performance in temporary automated driving. Therefore, the study aimed to investigate the effects of MR and voluntary NDRT on young drivers' fatigue and performance.

METHOD

Twenty-five young drivers experienced 60 min automated driving on a highway with low traffic density and a TOR prompted due to a collision event. A within-subjects was designed that comprised three conditions: NONE, TOR-only, and MR + TOR. Drivers were allowed to perform a self-paced phone NDRT during automated driving.

RESULTS

The PERCLOS and blink frequency data showed that playing phones could keep drivers vigilant. The take-over performance on whether taking phone had no difference, but with MRs condition exhibited better take-over performance including the shorter reaction time and the longer TTC. Subjective evaluations also showed the advantages of MRs with more safety, trust, acceptance, and lower workload.

CONCLUSIONS

Taking MRs had a positive effect on relieving fatigue and improving take-over performance. Furthermore, MRs could potentially improve the safety and acceptance of automated driving.

PRACTICAL APPLICATIONS

The MR design can be used in the automotive industry to ensure the safest interfaces between fatigue drivers and automation systems.

Evaluation of Imminent Take-Over Requests With Real Automation on a Test Track

OBJECTIVE

Investigating take-over, driving, non-driving related task (NDRT) performance, and trust of conditionally automated vehicles (AVs) in critical transitions on a test track.

BACKGROUND

Most experimental results addressing driver take-over were obtained in simulators. The presented experiment aimed at validating relevant findings while uncovering potential effects of motion cues and real risk.

METHOD

Twenty-two participants responded to four critical transitions on a test track. Non-driving related task modality (reading on a handheld device vs. auditory) and take-over timing (cognitive load) were varied on two levels. We evaluated take-over and NDRT performance as well as gaze behavior. Further, trust and workload were assessed with scales and interviews.

RESULTS

Reaction times were significantly faster than in simulator studies. Further, reaction times were only barely affected by varying visual, physical, or cognitive load. Post-take-over control was significantly degraded with the handheld device. Experiencing the system reduced participants' distrust, and distrusting participants monitored the system longer and more frequently. NDRTs on a handheld device resulted in more safety-critical situations.

CONCLUSION

The results confirm that take-over performance is mainly influenced by visual-cognitive load, while physical load did not significantly affect responses. Future take-over request (TOR) studies may investigate situation awareness and post-take-over control rather than reaction times only. Trust and distrust can be considered as different dimensions in AV research.

APPLICATION

Conditionally AVs should offer dedicated interfaces for NDRTs to provide an alternative to using nomadic devices. These interfaces should be designed in a way to maintain drivers' situation awareness.

PRÉCIS

This paper presents a test track experiment addressing conditionally automated driving systems. Twenty-two participants responded to critical TORs, where we varied NDRT modality and take-over timing. In addition, we assessed trust and workload with standardized scales and interviews.

Assessing the physiological effect of non-driving-related task performance and task modality in conditionally automated driving systems: A systematic review and meta-analysis

In conditionally automated driving, the driver is free to disengage from controlling the vehicle, but they are expected to resume driving in response to certain situations or events that the system is not equipped to respond to. As the level of vehicle automation increases, drivers often engage in non-driving-related tasks (NDRTs), defined as any secondary task unrelated to the primary task of driving. This engagement can have a detrimental effect on the driver's situation awareness and attentional resources. NDRTs with resource demands that overlap with the driving task, such as visual or manual tasks, may be particularly deleterious. Therefore, monitoring the driver's state is an important safety feature for conditionally automated vehicles, and physiological measures constitute a promising means of doing this. The present systematic review and meta-analysis synthesises findings from 32 studies concerning the effect of NDRTs on drivers' physiological responses, in addition to the effect of NDRTs with a visual or a manual modality. Evidence was found that NDRT engagement led to higher physiological arousal, indicated by increased heart rate, electrodermal activity and a decrease in heart rate variability. There was mixed evidence for an effect of both visual and manual NDRT modalities on all physiological measures. Understanding the relationship between task performance and arousal during automated driving is of critical importance to the development of driver monitoring systems and improving the safety of this technology.

Beyond adaptive cruise control and lane centering control: drivers' mental model of and trust in emerging ADAS technologies

Introduction

The potential safety benefits of advanced driver assistance systems (ADAS) highly rely on drivers' appropriate mental models of and trust in ADAS. Current research mainly focused on drivers' mental model of adaptive cruise control (ACC) and lane centering control (LCC), but rarely investigated drivers' understanding of emerging driving automation functions beyond ACC and LCC.

Methods

To address this research gap, 287 valid responses from ADAS users in the Chinese market, were collected in a survey study targeted toward state-of-the-art ADAS (e.g., autopilot in Tesla). Through cluster analysis, drivers were clustered into four groups based on their knowledge of traditional ACC and LCC functions, knowledge of functions beyond ACC and LCC, and knowledge of ADAS limitations. Predictors of driver grouping were analyzed, and we further modeled drivers' trust in ADAS.

Results

Drivers in general had weak knowledge of LCC functions and functions beyond ACC and LCC, and only 27 (9%) of respondents had a relatively strong mental model of ACC and LCC. At the same time, years of licensure, weekly driving distance, ADAS familiarity, driving style (i.e., planning), and personability (i.e., agreeableness) were associated with drivers' mental model of ADAS. Further, it was found that the mental model of ADAS, vehicle brand, and drivers' age, ADAS experience, driving style (i.e., focus), and personality (i.e., emotional stability) were significant predictors of drivers' trust in ADAS.

Discussion

These findings provide valuable insights for the design of driver education and training programs to improve driving safety with ADAS.

Impact of non-driving related tasks while operating automated driving systems (ADS): A systematic review

Automated Driving Systems (ADS) (SAE, 2021), promise improved safety and comfort for drivers. Current technological advances have resulted in increased automation capabilities. However, with the increase in automation capabilities, there is a shift in how drivers interact with their vehicles. Drivers can now temporarily hand over the control of the driving task to ADS under certain conditions. However, with ADS in temporary control of the vehicle, drivers may choose to engage in non-driving related tasks (NDRT). The current capabilities of ADS do not allow drivers to hand over control of the driving task indefinitely. Drivers must remain aware and be ready to take back control if necessary. There is a need to better understand drivers' performance and behaviors when driving with ADS, especially when engaged in NDRTs. This literature review, therefore, aims to understand the state of knowledge on automated vehicle systems and driver distraction. This review was conducted as per PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Studies found a significant increase in takeover times while engaging in NDRTs and driving with automation active. Studies also discuss a change in driver's visual attention, with more focus given to NDRTs as compared to the front roadway. The concerning effects of increasing reaction times and decreases in visual attention can be mitigated by using interventions and studies have had success in redirecting drivers attention and reorient them to the task of driving. The review, therefore, includes a discussion of ADS and NDRT engagement and its impact on driving behaviors such as take-over times, visual attention, trust, and workload. Implications on driver safety and performance are discussed in light of this synthesis.

A Study on the Entire Take-Over Process-Based Emergency Obstacle Avoidance Behavior

Nowadays, conditional automated driving vehicles still need drivers to take-over in the scenarios such as emergency hazard events or driving environments beyond the system's control. This study aimed to explore the changing trend of the drivers' takeover behavior under the influence of traffic density and take-over budget time for the entire take-over process in emergency obstacle avoidance scenarios. In the driving simulator, a 2 × 2 factorial design was adopted, including two traffic densities (high density and low density) and two kinds of take-over budget time (3 s and 5 s). A total of 40 drivers were recruited, and each driver was required to complete four simulation experiments. The driver's take-over process was divided into three phases, including the reaction phase, control phase, and recovery phase. Time parameters, dynamics parameters, and operation parameters were collected for each take-over phase in different obstacle avoidance scenarios. This study analyzed the variability of traffic density and take-over budget time with take-over time, lateral behavior, and longitudinal behavior. The results showed that in the reaction phase, the driver's reaction time became shorter as the scenario urgency increased. In the control phase, the steering wheel reversal rate, lateral deviation rate, braking rate, average speed, and takeover time were significantly different at different urgency levels. In the recovery phase, the average speed, accelerating rate, and take-over time differed significantly at different urgency levels. For the entire take-over process, the entire take-over time increased with the increase in urgency. The lateral take-over behavior tended to be aggressive first and then became defensive, and the longitudinal take-over behavior was defensive with the increase in urgency. The findings will provide theoretical and methodological support for the improvement of take-over behavior assistance in emergency take-over scenarios. It will also be helpful to optimize the human-machine interaction system.

Effects of unreliable automation, non-driving related task, and takeover time budget on drivers' takeover performance and workload

The objective of this study was to assess the effects of unreliable automation, non-driving related tasks (NDRTs), and takeover time budget (TOTB) on drivers' takeover performance and cognitive workload when faced with critical incidents. Automated vehicles are expected to improve traffic safety. However, there are still some concerns about the effects of automation failures on driver performance and workload. Twenty-eight drivers participated in a driving simulation study. The findings suggested that drivers require at least 8 s of TOTB to safely take over the control of the vehicle. In addition, drivers exhibited safer takeover performance under the conditionally automated driving situation than negotiating the critical incident in the manual driving condition. The results of drivers' cognitive workload were inconclusive, which might be due to the individual and recall biases in subjective measures that could not capture subtle differences in workload during takeover requests.Practitioner Summary: A driving simulation study was conducted to assess the effect of unreliable automation, non-driving related tasks, and different takeover time budgets on drivers' performance and workload. The results can provide guidelines for vehicle manufacturers to improve the design of automated vehicles.

A Systematic Review and Meta-Analysis of Takeover Performance During Conditionally Automated Driving

OBJECTIVE

The aim of this paper was to synthesize the experimental research on factors that affect takeover performance during conditionally automated driving.

BACKGROUND

For conditionally automated driving, the automated driving system (ADS) can handle the entire dynamic driving task but only for limited domains. When the system reaches a limit, the driver is responsible for taking over vehicle control, which may be affected by how much time they are provided to take over, what they were doing prior to the takeover, or the type of information provided to them during the takeover.

METHOD

Out of 8446 articles identified by a systematic literature search, 48 articles containing 51 experiments were included in the meta-analysis. Coded independent variables were time budget, non-driving related task engagement and resource demands, and information support during the takeover. Coded dependent variables were takeover timing and quality measures.

RESULTS

Engaging in non-driving related tasks results in degraded takeover performance, particularly if it has overlapping resource demands with the driving task. Weak evidence suggests takeover performance is impaired with shorter time budgets. Current implementations of information support did not affect takeover performance.

CONCLUSION

Future research and implementation should focus on providing the driver more time to take over while automation is active and should further explore information support.

APPLICATION

The results of the current paper indicate the need for the development and deployment of vehicle-to-everything (V2X) services and driver monitoring.

Drivers of partially automated vehicles are blamed for crashes that they cannot reasonably avoid

People seem to hold the human driver to be primarily responsible when their partially automated vehicle crashes, yet is this reasonable? While the driver is often required to immediately take over from the automation when it fails, placing such high expectations on the driver to remain vigilant in partially automated driving is unreasonable. Drivers show difficulties in taking over control when needed immediately, potentially resulting in dangerous situations. From a normative perspective, it would be reasonable to consider the impact of automation on the driver's ability to take over control when attributing responsibility for a crash. We, therefore, analyzed whether the public indeed considers driver ability when attributing responsibility to the driver, the vehicle, and its manufacturer. Participants blamed the driver primarily, even though they recognized the driver's decreased ability to avoid the crash. These results portend undesirable situations in which users of partially driving automation are the ones held responsible, which may be unreasonable due to the detrimental impact of driving automation on human drivers. Lastly, the outcome signals that public awareness of such human-factors issues with automated driving should be improved.

A user study of directional tactile and auditory user interfaces for take-over requests in conditionally automated vehicles

General introduction of unconditionally and conditionally automated vehicles is expected to have a highly positive impact on the society, from increased accessibility to mobility and road traffic safety, to decreased environmental and economic negative impacts. However, there are several obstacles and risks slowing down the adoption of this technology, which are primarily related to the human-machine interaction (HMI) and exchange of control between the vehicle and the human driver. In this article, we present key takeaways for HMI design of take-over requests (TOR) that the vehicle issues to inform the driver to take over control of the vehicle. The key takeaways were developed based on the results of a user study, where directional tactile-ambient (visual) and auditory-ambient (visual) TOR user interfaces (UI) were evaluated with regards to commonly used take-over quality aspects (attention redirection, take-over time, correct interpretation of stimuli, off-road drive, brake application, lateral acceleration, minimal time-to-collision and occurrence of collision). 36 participants took part in the mixed design study, which was conducted in a driving simulator. The results showed that drivers' attention was statistically significantly faster redirected with the auditory-ambient UI, however using the tactile-ambient UI resulted in less off-road driving and slightly less collisions. The results also revealed that drivers correctly interpreted the directional TOR stimuli more often than the non-directional one. Based on the study results, a list of key takeaways was developed and is presented in the conclusion of the paper. The results from this study are especially relevant to the TOR UI designers and the automotive industry, which tend to provide the most usable UI for ensuring safer end efficient human-vehicle interaction during the TOR task.

Analysing the effect of gender on the human-machine interaction in level 3 automated vehicles

The emergence of the level 3 automated vehicles (L3 AVs) can enable drivers to be completely disengaged from driving and safely perform other non-driving related tasks, but sometimes their takeover of control of the vehicle is required. The takeover of control is an important human–machine interaction in L3 AVs. However, little research has focused on investigating the effect of gender on takeover performance. In order to fill this research gap, a driving simulator study with 76 drivers (33 females and 43 males) was conducted. The participants took over control from L3 AVs, and the timing and quality of takeover were measured. The results show that although there was no significant difference in most of the measurements adopted to quantify takeover performance between female and male. Gender did affect takeover performance slightly, with women exhibited slightly better performance than men. Compared to men, women exhibited a smaller percentage of hasty takeovers and slightly faster reaction times as well as slightly more stable operation of the steering wheel. The findings highlight that it is important for both genders to recognise they can use and interact with L3 AVs well, and more hands-on experience and teaching sessions could be provided to deepen their understanding of L3 AVs. The design of the car interiors of L3 AVs should also take into account gender differences in the preferences of users for different non-driving related tasks.

How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review

Drowsiness and distraction are major factors of road crashes and responsible of>35% of road fatalities. Automated driving could solve or minimize their impact, yet it is also in itself a way to promote them. Previous literature reviews and meta-analysis regarding take-overs during automated driving primarily focused on distraction rather than drowsiness. We thus present a systematic and meta-analysis literature review focused on the effect of distraction and drowsiness on take-over performance. From an initial selection of 1896 articles from databases, we obtained by applying systematic review methodology a total of 58 articles with 42 articles dedicated to distraction and 17 articles related to drowsiness. According to our analysis, we demonstrated that distraction and drowsiness increased the take-over request reaction time (TOR-RT), which could also lead to a reduction of the quality of take-overs. In addition, this longer reaction time was even more important in the case of handheld non-driving related tasks, which is important to consider as phone use is among the most frequent tasks done during automated driving. On a more methodological aspect, we also demonstrated that take-over time budget had a significant effect on TOR-RT. However, it is difficult to estimate to what extend distraction and drowsiness could impact the take-over quality, even if several elements supported safety issues. We underpinned several limits of the current methodologies applied in the study of distraction and drowsiness such as (i) the lack of additional measures related to the take-over quality (e.g., accelerations, collision rate), (ii) the many different methodologies applied to the determination of the TOR-RT (e.g., deactivation by the steering wheel, pedals, button), (iii) the high frequency of take-over requests which can lead to habituation effects, (iv) the lack of control conditions, (v) the fact that the level of drowsiness was relatively low in most studies. We thus highlighted recommendations for a better estimation of the effect of distraction and drowsiness on take-over performance. Further studies should adopt more standardized measures of TOR-RT and additional take-over quality measures, try minimizing the number of take-over requests, and carefully consider the time budget available for the use case since it influences the TOR-RT. Regarding distraction, researchers should consider the impact of tasks requiring handholding items. Concerning drowsiness, further protocols should consider the non-linearity of drowsiness and presence of micro sleeps and favor take-over requests based on drowsiness level protocols rather than on fixed duration protocols.

Transitions Between Highly Automated and Longitudinally Assisted Driving: The Role of the Initiator in the Fight for Authority

OBJECTIVE

A driving simulator study explored how drivers behaved depending on their initial role during transitions between highly automated driving (HAD) and longitudinally assisted driving (via adaptive cruise control).

BACKGROUND

During HAD, drivers might issue a take-over request (TOR), initiating a transition of control that was not planned. Understanding how drivers behave in this situation and, ultimately, the implications on road safety is of paramount importance.

METHOD

Sixteen participants were recruited for this study and performed transitions of control between HAD and longitudinally assisted driving in a driving simulator. While comparing how drivers behaved depending on whether or not they were the initiators, different handover strategies were presented to analyze how drivers adapted to variations in the authority level they were granted at various stages of the transitions.

RESULTS

Whenever they initiated the transition, drivers were more engaged with the driving task and less prone to follow the guidance of the proposed strategies. Moreover, initiating a transition and having the highest authority share during the handover made the drivers more engaged with the driving task and attentive toward the road.

CONCLUSION

Handover strategies that retained a larger authority share were more effective whenever the automation initiated the transition. Under driver-initiated transitions, reducing drivers' authority was detrimental for both performance and comfort.

APPLICATION

As the operational design domain of automated vehicles (Society of Automotive Engineers [SAE] Level 3/4) expands, the drivers might very well fight boredom by taking over spontaneously, introducing safety issues so far not considered but nevertheless very important.

The effects of takeover request lead time on drivers' situation awareness for manually exiting from freeways: A web-based study on level 3 automated vehicles

Conditional automation systems allow drivers to turn their attention away from the driving task in certain scenarios but still require drivers to gain situation awareness (SA) upon a takeover request (ToR) and resume manual control when the system is unable to handle the upcoming situation. Unlike time-critical takeover situations in which drivers must respond within a relatively short time frame, the ToRs for non-critical events such as exiting from a freeway can be scheduled way ahead of time. It is unknown how the ToR lead time affects driver SA for resuming manual control and when to send the ToR is most appropriate in non-critical takeover events. The present study conducted a web-based, supervised experiment with 31 participants using conditional automation systems in freeway existing scenarios while playing a mobile game. Each participant experienced 12 trials with different ToR lead times (6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 45, and 60 s) for exiting from freeways in a randomized order. Driver SA was measured by using a freeze probe technique in each trial when the participant pressed the spacebar on the laptop to simulate the takeover action. Results revealed a positive effect of longer ToR lead times on driver SA for resuming control to exit from freeways and the effect leveled off at the lead time of 16-30 s. The participants tended to postpone their takeover actions further when they were given a longer ToR lead time and it did not level off up to 60 s. Nevertheless, not all drivers waited till the last moment to take over AVs even though they did not get sufficient SA. The ToR lead time of 16-30 s was recommended for better SA; and it could be narrowed down to 25-30 s if considering the subjective evaluations on takeover readiness, workload, and trust. The findings provide implications for the future design of conditional automation systems used for freeway driving.

Age-related effects of executive function on takeover performance in automated driving

The development of highly automated vehicles can meet elderly drivers' mobility needs; however, worse driving performance after a takeover request (TOR) is frequently found, especially regarding non-driving related tasks (NDRTs). This study aims to detect the correlation between takeover performance and underlying cognitive factors comprising a set of higher order cognitive processes including executive functions. Thirty-five young and 35 elderly participants were tested by computerized cognitive tasks and simulated driving tasks to evaluate their executive functions and takeover performance. Performance of n-back tasks, Simon tasks, and task switching were used to evaluate updating, inhibition, and shifting components of executive functions by principal component analysis. The performance of lane changing after TOR was measured using the standard deviation of the steering wheel angle and minimum time-to-collision (TTC). Differences between age groups and NDRT engagement were assessed by two-way mixed analysis of variance. Older participants had significantly lower executive function ability and were less stable and more conservative when engaged in NDRT. Furthermore, a significant correlation between executive function and lateral driving stability was found. These findings highlight the interaction between age-related differences in executive functions and takeover performance; thus, provide implications for designing driver screening tests or human-machine interfaces.

The effect of inconsistent steering guidance during transitions from Highly Automated Driving

This driving simulator study investigated the effect of inconsistent steering guidance during system and user-initiated transitions from Highly Automated Driving (HAD). In particular, the aim of the study was to understand if steering conflicts could be achieved by adopting inconsistent steering guidance and whether these conflicts could be exploited to accelerate drivers' steering engagement within a limited time. Inconsistent steering guidance was generated by switching the guidance on and off at 3 different frequencies (0.1, 0.2 and 0.3 Hz). Results revealed that steering engagement has more to do with the initiation rather than the quality of the steering guidance. In fact, drivers were more engaged with the steering task when they initiated the transition themselves. Compared to system-initiated transitions, in user-initiated ones, drivers exerted stronger steering inputs throughout the transition, which allowed them to maintain larger Time To Lane Crossing (TTLC) values with fewer steering corrections. During system-initiated transitions, drivers started to actively engage with the steering activity only after more than 5 s from the start of the transition but were able to achieve a steering behaviour close to the one shown during user-initiated transitions at 10 s.

Take-Over Requests after Waking in Autonomous Vehicles

Autonomous vehicles (AVs) enable drivers to devote their primary attention to non-driving-related tasks (NDRTs). Consequently, AVs must provide intelligibility services appropriate to drivers’ in-situ states and in-car activities to ensure driver safety, and accounting for the type of NDRT being performed can result in higher intelligibility. We discovered that sleeping is drivers’ most preferred NDRT, and this could also result in a critical scenario when a take-over request (TOR) occurs. In this study, we designed TOR situations where drivers are woken from sleep in a high-fidelity AV simulator with motion systems, aiming to examine how drivers react to a TOR provided with our experimental conditions. We investigated how driving performance, perceived task workload, AV acceptance, and physiological responses in a TOR vary according to two factors: (1) feedforward timings and (2) presentation modalities. The results showed that when awakened by a TOR alert delivered >10 s prior to an event, drivers were more focused on the driving context and were unlikely to be influenced by TOR modality, whereas TOR alerts delivered <5 s prior needed a visual accompaniment to quickly inform drivers of on-road situations. This study furthers understanding of how a driver’s cognitive and physical demands interact with TOR situations at the moment of waking from sleep and designs effective interventions for intelligibility services to best comply with safety and driver experience in AVs.

The Identification of Non-Driving Activities with Associated Implication on the Take-Over Process

In conditionally automated driving, the engagement of non-driving activities (NDAs) can be regarded as the main factor that affects the driver’s take-over performance, the investigation of which is of great importance to the design of an intelligent human–machine interface for a safe and smooth control transition. This paper introduces a 3D convolutional neural network-based system to recognize six types of driver behaviour (four types of NDAs and two types of driving activities) through two video feeds based on head and hand movement. Based on the interaction of driver and object, the selected NDAs are divided into active mode and passive mode. The proposed recognition system achieves 85.87% accuracy for the classification of six activities. The impact of NDAs on the perspective of the driver’s situation awareness and take-over quality in terms of both activity type and interaction mode is further investigated. The results show that at a similar level of achieved maximum lateral error, the engagement of NDAs demands more time for drivers to accomplish the control transition, especially for the active mode NDAs engagement, which is more mentally demanding and reduces drivers’ sensitiveness to the driving situation change. Moreover, the haptic feedback torque from the steering wheel could help to reduce the time of the transition process, which can be regarded as a productive assistance system for the take-over process.

Effects of Distraction in On-Road Level 2 Automated Driving: Impacts on Glance Behavior and Takeover Performance

OBJECTIVE

The paper aimed to investigate glance behaviors under different levels of distraction in automated driving (AD) and understand the impact of distraction levels on driver takeover performance.

BACKGROUND

Driver distraction detrimentally affects takeover performance. Glance-based distraction measurement could be a promising method to remind drivers to maintain enough attentiveness before the takeover request in partially AD.

METHOD

Thirty-six participants were recruited to drive a Tesla Model S in manual and Autopilot modes on a test track while engaging in secondary tasks, including temperature-control, email-sorting, and music-selection, to impose low and high distractions. During the test drive, participants needed to quickly change the lane as if avoiding an immediate road hazard if they heard an unexpected takeover request (an auditory warning). Driver state and behavior over the test drive were recorded in real time by a driver monitoring system and several other sensors installed in the Tesla vehicle.

RESULTS

The distribution of off-road glance duration was heavily skewed (with a long tail) by high distractions, with extreme glance duration more than 30 s. Moreover, being eyes-off-road before takeover could cause more delay in the urgent takeover reaction compared to being hands-off-wheel.

CONCLUSION

The study measured off-road glance duration under different levels of distraction and demonstrated the impacts of being eyes-off-road and hands-off-wheel on the following takeover performance.

APPLICATION

The findings provide new insights about engagement in Level 2 AD and are useful for the design of driver monitoring technologies for distraction management.

User Monitoring in Autonomous Driving System Using Gamified Task: A Case for VR/AR In-Car Gaming

Background: As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver’s obligation will be changed to a supervisory role. A key point to consider is the driver’s engagement in the secondary task to maintain the driver/user in the control loop. This paper aims to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. Methods: We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. Results: There was a significant difference in the tasks (F(2,33) = 4.34, p = 0.0213), identifying the game-task as significant with respect to reaction time and ideal for the present investigation. Game scoring followed three profiles/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify/infer drivers’ engagement with the game task. Conclusion: The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments with AR-Games focusing on the real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.

Intelligent Mobility in the City: The Influence of System and Context Factors on Drivers’ Takeover Willingness and Trust in Automated Vehicles

Automated driving in urban environments not only has the potential to improve traffic flow and heighten driver comfort but also to increase traffic safety, particularly for vulnerable road users such as pedestrians. For these benefits to take effect, drivers need to trust and use automated vehicles. This decision is influenced by both system and context factors. However, it is not yet clear how these factors interact with each other, especially for automated driving in city scenarios with crossing pedestrians. Therefore, we conducted an online experiment in which participants (N = 68) experienced short automated rides from the driver’s perspective through an urban environment. In each of the presented videos, a pedestrian crossed the street in front of the automated vehicle while system and context factors were varied: 1) the crossing pedestrian’s intention was either visualized correctly (as crossing) or incorrectly (visualization missing) by the automated vehicle (system factor), 2) the pedestrian was either distracted by using a smartphone while crossing or not (context factor), and 3) the scenario was either more or less complex depending on the number of other vehicles and pedestrians being present (context factor). In situations with a system malfunction where the crossing pedestrian’s intention was not visualized, participants perceived the situation as more critical, had less trust in the automated system, and a higher willingness to take over control regardless of any context factors. However, when the system worked correctly, the crossing pedestrian’s smartphone usage came into play, especially in the less complex scenario. Participants perceived situations with a distracted pedestrian as more critical, trusted the system less, indicated a higher willingness to take over control, and were more uncertain about their decision. As this study demonstrates the influence of distracted pedestrians, more research is needed on context factors and their inclusion in the design of interfaces to keep drivers informed during automated driving in urban environments.

Designing for the Extremes: Modeling Drivers' Response Time to Take Back Control From Automation Using Bayesian Quantile Regression

OBJECTIVE

Understanding the factors that affect drivers' response time in takeover from automation can help guide the design of vehicle systems to aid drivers. Higher quantiles of the response time distribution might indicate a higher risk of an unsuccessful takeover. Therefore, assessments of these systems should consider upper quantiles rather than focusing on the central tendency.

BACKGROUND

Drivers' responses to takeover requests can be assessed using the time it takes the driver to take over control. However, all the takeover timing studies that we could find focused on the mean response time.

METHOD

A study using an advanced driving simulator evaluated the effect of takeover request timing, event type at the onset of a takeover, and visual demand on drivers' response time. A mixed effects model was fit to the data using Bayesian quantile regression.

RESULTS

Takeover request timing, event type that precipitated the takeover, and the visual demand all affect driver response time. These factors affected the 85th percentile differently than the median. This was most evident in the revealed stopped vehicle event and conditions with a longer time budget and scenes with lower visual demand.

CONCLUSION

Because the factors affect the quantiles of the distribution differently, a focus on the mean response can misrepresent actual system performance. The 85th percentile is an important performance metric because it reveals factors that contribute to delayed responses and potentially dangerous outcomes, and it also indicates how well the system accommodates differences between drivers.

Updating our understanding of situation awareness in relation to remote operators of autonomous vehicles

The introduction of autonomous vehicles (AVs) could prevent many accidents attributable to human driver error. However, even entirely driverless vehicles will sometimes require remote human intervention. Current taxonomies of automated driving do not acknowledge the possibility of remote control of AVs or the challenges that are unique to such a driver in charge of a vehicle that they are not physically occupying. Yet there are significant differences between situation awareness (SA) in normal driving contexts and SA in these remote driving operations. We argue that the established understanding of automated driving requires updating to include the context of remote operation that is likely to come in to play at higher levels of automation. It is imperative to integrate the role of the remote operator within industry standard taxonomies, so that regulatory frameworks can be established with regards to the training required for remote operation, the necessary equipment and technology, and a comprehensive inventory of the use cases under which we could expect remote operation to be carried out. We emphasise the importance of designing control interfaces in a way that will maximise remote operator (RO) SA and we identify some principles for designing systems aimed at increasing an RO's sense of embodiment in the AV that requires temporary control.

Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs

Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver. This paper applies a user-centred design approach to designing a new AV interior to better support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with 44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were identified as the principal concerns about future autonomous vehicles. Based on these findings, Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information display to facilitate users' activities and situational awareness while driving. Study 3 consists of a series of laboratory simulator evaluation studies, and this indicated that drivers in the proposed design condition had better situational awareness in an AV when dealing with take-over situations. Such findings suggest the possibility of applying rear-facing seats in autonomous vehicles to support in-vehicle non-driving activities. Some specific implications of designs to enhance a driver's situational awareness have also been suggested.

Automated Driving: A Literature Review of the Take over Request in Conditional Automation

In conditional automation (level 3), human drivers can hand over the Driving Dynamic Task (DDT) to the Automated Driving System (ADS) and only be ready to resume control in emergency situations, allowing them to be engaged in non-driving related tasks (NDRT) whilst the vehicle operates within its Operational Design Domain (ODD). Outside the ODD, a safe transition process from the ADS engaged mode to manual driving should be initiated by the system through the issue of an appropriate Take Over Request (TOR). In this case, the driver’s state plays a fundamental role, as a low attention level might increase driver reaction time to take over control of the vehicle. This paper summarizes and analyzes previously published works in the field of conditional automation and the TOR process. It introduces the topic in the appropriate context describing as well a variety of concerns that are associated with the TOR. It also provides theoretical foundations on implemented designs, and report on concrete examples that are targeted towards designers and the general public. Moreover, it compiles guidelines and standards related to automation in driving and highlights the research gaps that need to be addressed in future research, discussing also approaches and limitations and providing conclusions.

Predicting driver takeover performance in conditionally automated driving

In conditionally automated driving, drivers have difficulty taking over control when requested. To address this challenge, we aimed to predict drivers' takeover performance before the issue of a takeover request (TOR) by analyzing drivers' physiological data and external environment data. We used data sets from two human-in-the-loop experiments, wherein drivers engaged in non-driving-related tasks (NDRTs) were requested to take over control from automated driving in various situations. Drivers' physiological data included heart rate indices, galvanic skin response indices, and eye-tracking metrics. Driving environment data included scenario type, traffic density, and TOR lead time. Drivers' takeover performance was categorized as good or bad according to their driving behaviors during the transition period and was treated as the ground truth. Using six machine learning methods, we found that the random forest classifier performed the best and was able to predict drivers' takeover performance when they were engaged in NDRTs with different levels of cognitive load. We recommended 3 s as the optimal time window to predict takeover performance using the random forest classifier, with an accuracy of 84.3% and an F1-score of 64.0%. Our findings have implications for the algorithm development of driver state detection and the design of adaptive in-vehicle alert systems in conditionally automated driving.

Predicting takeover response to silent automated vehicle failures

Current and foreseeable automated vehicles are not able to respond appropriately in all circumstances and require human monitoring. An experimental examination of steering automation failure shows that response latency, variability and corrective manoeuvring systematically depend on failure severity and the cognitive load of the driver. The results are formalised into a probabilistic predictive model of response latencies that accounts for failure severity, cognitive load and variability within and between drivers. The model predicts high rates of unsafe outcomes in plausible automation failure scenarios. These findings underline that understanding variability in failure responses is crucial for understanding outcomes in automation failures.

Task Interruption and Control Recovery Strategies After Take-Over Requests Emphasize Need for Measures of Situation Awareness

OBJECTIVE

Our objective was to determine whether there is a need to go beyond measures of automation deactivation time to understand the transition to manual driving after take-over requests (TORs) using the example of office tasks as nondriving-related tasks (NDRTs).

BACKGROUND

Office tasks are likely NDRTs during automated commutes to/from work. Complex tasks can influence how manual control and visual attention is recovered after TORs.

METHOD

N = 51 participants in a driving simulator performed either one of two office tasks or no task (between subjects). We recorded reaction times in a high-urgency and low-urgency scenario (within subjects) and analyzed task interruption strategies.

RESULTS

90% of the participants who performed an NDRT deactivated the automation after 7 to 8 s. However, 90% of the same drivers looked at the side mirror for the first time only after 11 to 14 s. Drivers with office tasks either interrupted the tasks sequentially or in parallel. Strategies were not adapted to the take-over situation or the task but appeared to be due to individual preferences.

CONCLUSION

Drivers engaged in NDRTs may neglect lower priority subtasks after a TOR, such as mirror checking. Therefore, there is a need to go beyond measures of automation deactivation time to understand the transition to manual driving. Using analyses of attentional dynamics during take-over situations may enhance the safety of future car-driver handover assistance systems.

APPLICATION

If low driver availability is detected, TORs should only be used as a fallback option if sufficient time and adaptive driver support can be provided.

What happens when drivers of automated vehicles take over control in critical brake situations?

Even with automated vehicles, driving situations with short time headways and extreme vehicle dynamics may arise when unpredictable events occur. If drivers take back control under such conditions, it is uncertain how they behave and how well they can cope with the situation. This issue has not been investigated yet and is subject to our study. In a driving simulator, non-distracted participants (N = 42) experienced nine critical situations caused by a braking vehicle in front of them. Time headway and longitudinal vehicle dynamics were varied to create different degrees of objective criticality. Participants' criticality ratings, take-over behavior, and driving performance were recorded and analyzed. The results indicate that participants were sensitive to changes in objective criticality and adapted their behavior. Take-over times were very fast under all conditions and participants showed higher criticality ratings, more intense decelerations, and more lane changes with increasing objective criticality. To avoid a collision, participants decelerated much more than the automation and changed lanes, even though this was not necessary. Thereby, they raised the risk of vehicle instability, rear-end collisions, and collisions with overtaking vehicles. To conclude, take-overs in critical brake situations may be a threat to the safety of drivers and other road users because drivers' reactions are more pronounced than necessary. These results suggest that assistive functions are required to support drivers in critical take-over situations.

Effects of cognitive and visual loads on driving performance after take-over request (TOR) in automated driving

The present study investigated effects of cognitive and visual loads on driving performance after take-over request (TOR) in an automated driving task. Participants completed automated driving in a driving simulator without a non-driving related task, with an easy non-driving related task, and with a difficult non-driving related task. The primary task was to monitor the environment and the system state. An N-back task and a Surrogate Reference Task (SuRT) were adapted to induce cognitive and visual loads respectively. The system followed a front vehicle automatically. Driving performance was measured by responses to a critical event (appearance of a broken-down car) after the automated system issued TOR and then terminated. High subjective difficulty of the N-back task was related to increased time and increased steering angle variance in the time course from onset of steering control to lane change, while high subjective difficulty of SuRT was related to increased steering angle variance in the time course after lane change. This suggests that both cognitive and visual loads affect driving performance after TOR in automated driving, but the effects appear in different time courses.

Engagement in Non-Driving Related Tasks as a Non-Intrusive Measure for Mode Awareness: A Simulator Study

Research on the role of non-driving related tasks (NDRT) in the area of automated driving is indispensable. At the same time, the construct mode awareness has received considerable interest in regard to human–machine interface (HMI) evaluation. Based on the expectation that HMI design and practice with different levels of driving automation influence NDRT engagement, a driving simulator study was conducted. In a 2 × 5 (automation level x block) design, N = 49 participants completed several transitions of control. They were told that they could engage in an NDRT if they felt safe and comfortable to do so. The NDRT was the Surrogate Reference Task (SuRT) as a representative of a wide range of visual–manual NDRTs. Engagement (i.e., number of inputs on the NDRT interface) was assessed at the onset of a respective episode of automated driving (i.e., after transition) and during ongoing automation (i.e., before subsequent transition). Results revealed that over time, NDRT engagement increased during both L2 and L3 automation until stable engagement at the third block. This trend was observed for both onset and ongoing NDRT engagement. The overall engagement level and the increase in engagement are significantly stronger for L3 automation compared to L2 automation. These results outline the potential of NDRT engagement as an online non-intrusive measure for mode awareness. Moreover, repeated interaction is necessary until users are familiar with the automated system and its HMI to engage in NDRTs. These results provide researchers and practitioners with indications about users’ minimum degree of familiarity with driving automation and HMIs for mode awareness testing.

The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios

In the development of highly automated driving systems (L3 and 4), much research has been done on the subject of driver takeover. Strong focus has been placed on the takeover quality. Previous research has shown that one of the main influencing factors is the complexity of a traffic situation that has not been sufficiently addressed so far, as different approaches towards complexity exist. This paper differentiates between the objective complexity and the subjectively perceived complexity. In addition, the familiarity with a takeover situation is examined. Gold et al. show that repetition of takeover scenarios strongly influences the take-over performance. Yet, both complexity and familiarity have not been considered at the same time. Therefore, the aim of the present study is to examine the impact of objective complexity and familiarity on the subjectively perceived complexity and the resulting takeover quality. In a driving simulator study, participants are requested to take over vehicle control in an uncritical situation. Familiarity and objective complexity are varied by the number of surrounding vehicles and scenario repetitions. Subjective complexity is measured using the NASA-TLX; the takeover quality is gathered using the take-over controllability rating (TOC-Rating). The statistical evaluation results show that the parameters significantly influence the takeover quality. This is an important finding for the design of cognitive assistance systems for future highly automated and intelligent vehicles.

Detecting Driver’s Fatigue, Distraction and Activity Using a Non-Intrusive Ai-Based Monitoring System

The lack of attention during the driving task is considered as a major risk factor for fatal road accidents around the world. Despite the ever-growing trend for autonomous driving which promises to bring greater road-safety benefits, the fact is today’s vehicles still only feature partial and conditional automation, demanding frequent driver action. Moreover, the monotony of such a scenario may induce fatigue or distraction, reducing driver awareness and impairing the regain of the vehicle’s control. To address this challenge, we introduce a non-intrusive system to monitor the driver in terms of fatigue, distraction, and activity. The proposed system explores state-of-the-art sensors, as well as machine learning algorithms for data extraction and modeling. In the domain of fatigue supervision, we propose a feature set that considers the vehicle’s automation level. In terms of distraction assessment, the contributions concern (i) a holistic system that covers the full range of driver distraction types and (ii) a monitoring unit that predicts the driver activity causing the faulty behavior. By comparing the performance of Support Vector Machines against Decision Trees, conducted experiments indicated that our system can predict the driver’s state with an accuracy ranging from 89% to 93%.

Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures

OBJECTIVE

This article provides a review of empirical studies of automated vehicle takeovers and driver modeling to identify influential factors and their impacts on takeover performance and suggest driver models that can capture them.

BACKGROUND

Significant safety issues remain in automated-to-manual transitions of vehicle control. Developing models and computer simulations of automated vehicle control transitions may help designers mitigate these issues, but only if accurate models are used. Selecting accurate models requires estimating the impact of factors that influence takeovers.

METHOD

Articles describing automated vehicle takeovers or driver modeling research were identified through a systematic approach. Inclusion criteria were used to identify relevant studies and models of braking, steering, and the complete takeover process for further review.

RESULTS

The reviewed studies on automated vehicle takeovers identified several factors that significantly influence takeover time and post-takeover control. Drivers were found to respond similarly between manual emergencies and automated takeovers, albeit with a delay. The findings suggest that existing braking and steering models for manual driving may be applicable to modeling automated vehicle takeovers.

CONCLUSION

Time budget, repeated exposure to takeovers, silent failures, and handheld secondary tasks significantly influence takeover time. These factors in addition to takeover request modality, driving environment, non-handheld secondary tasks, level of automation, trust, fatigue, and alcohol significantly impact post-takeover control. Models that capture these effects through evidence accumulation were identified as promising directions for future work.

APPLICATION

Stakeholders interested in driver behavior during automated vehicle takeovers may use this article to identify starting points for their work.

Noncritical State Transitions During Conditionally Automated Driving on German Freeways: Effects of Non-Driving Related Tasks on Takeover Time and Takeover Quality

OBJECTIVE

This study aimed at investigating the driver's takeover performance when switching from working on different non-driving related tasks (NDRTs) while driving with a conditionally automated driving function (SAE L3), which was simulated by a Wizard of Oz vehicle, to manual vehicle control under naturalistic driving conditions.

BACKGROUND

Conditionally automated driving systems, which are currently close to market introduction, require the user to stay fallback ready. As users will be allowed to engage in more complex NDRTs during the automated drive than when driving manually, the time needed to regain full manual control could likely be increased.

METHOD

Thirty-four users engaged in different everyday NDRTs while driving automatically with a Wizard of Oz vehicle. After approximately either 5 min or 15 min of automated driving, users were requested to take back vehicle control in noncritical situations. The test drive took place in everyday traffic on German freeways in the metropolitan area of Stuttgart.

RESULTS

Particularly tasks that required users to turn away from the central road scene or hold an object in their hands led to increased takeover times. Accordingly, increased variance in the driver's lane position was found shortly after the switch to manual control. However, the drivers rated the takeover situations to be mostly "harmless."

CONCLUSION

Drivers managed to regain control over the vehicle safely, but they needed more time to prepare for the manual takeover when the NDRTs caused motoric workload.

APPLICATION

The timings found in the study can be used to design comfortable and safe takeover concepts for automated vehicles.

The effects of takeover request modalities on highly automated car control transitions

This study investigated the influences of takeover request (TOR) modalities on a drivers' takeover performance after they engaged in non-driving related (NDR) tasks in highly automated driving (HAD). Visual, vibrotactile, and auditory modalities were varied in the design of the experiment under four conditions: no-task, phone conversation, smartphone interaction, and video watching tasks. Driving simulator experiments were conducted to analyze the drivers' take-over performance by collecting data during the transition time of re-engaging control of the vehicle, the time taken to be on the loop, and time taken to be physically ready to drive. Data were gathered on the perceived usefulness, safety, satisfaction, and effectiveness for each TOR based on a self-reported questionnaire. Takeover and hands-on times varied considerably, as shown by high standard deviation values between modalities, especially for phone conversations and smartphone interaction tasks. Moreover, it was found that participants failed to take over control of the vehicle when they were given visual TORs for phone conversation and smartphone interaction tasks. The perceived safety and satisfaction varied for the NDR task. Results from the statistical analysis showed that the NDR task significantly influenced the takeover time, but there was no significant interaction effect between the TOR modalities and the NDR task. The results could potentially be applied to the design of safe and efficient transitions of highly controlled, automated driving, where drivers are enabled to engage in NDR tasks.

From partial and high automation to manual driving: Relationship between non-driving related tasks, drowsiness and take-over performance

BACKGROUND

Until the level of full vehicle automation is reached, users of vehicle automation systems will be required to take over manual control of the vehicle occasionally and stay fallback-ready to some extent during the drive. Both, drowsiness caused by inactivity and the engagement in distracting non-driving related tasks (NDRTs) such as entertainment or office work have been suggested to impair the driver's ability to safely handle these transitions of control. Thus, it is an open question whether engagement in NDRTs will impair or improve take-over performance.

METHOD

In a motion-based driving simulator, 64 participants completed an automated drive that lasted either one or two hours using either a partially or highly automated driving system. In the partially automated driving condition, a warning was issued after several seconds when drivers took both hands off the steering wheel, while the highly automated driving system allowed hands-off driving permanently. Drivers were allowed to bring along their smartphones and to use them during the drive. They engaged in a wide variety of NDRTs such as reading or using social media. At the end of the session, drivers had to react to a sudden lead vehicle braking event. In the partial automation condition, there was no take-over request (TOR) to notify the drivers of the braking vehicle, while in the highly automated condition, the situation happened right after the drivers had deactivated the automation in response to a TOR. The lead time of the TOR was set at 8 s. Driver's level of drowsiness, workload (visual, mental and motoric) from carrying out the NDRT and motivational appeal of the NDRT right before the control transition were video-coded and used to predict the outcome of the braking event (i.e., reaction and system deactivation times, minimal Time-to-collision (TTC) and self-reported criticality) with a multiple regression approach.

RESULTS

In the partial automation condition, reaction times to the braking vehicle and situation criticality as measured by the minimum TTC could be well predicted. Main predictors for increased reaction time were drowsiness and motivational appeal of the NDRT. However, visual and mental demand associated with NDRTs did decrease reaction time, suggesting that the NDRT helped the drivers to maintain alertness during the partially automated drive. Accordingly, drowsiness and motivational appeal of the NDRT increased situation criticality, while cognitive load due to the NDRT decreased it. In the highly automated condition, however, it was not possible to predict system deactivation time (in reaction to the TOR), brake reaction time to the braking vehicle and situation criticality by observed drowsiness and NDRT engagement.

DISCUSSION

The results suggest a relationship between the driver's drowsiness and NDRT engagement in partial automation but not in highly automated driving. Several explanations for this finding are discussed. It could be possible that the lead time of 8 s might have given the drivers enough time to complete the driver state transition process from executing NDRTs to manual driving, putting them in a position to be able to cope with the driving event, while this was not possible in the partial automation condition. Methodological issues that might have led to a non-detection of an effect of drowsiness or NDRT engagement in the highly automated driving condition, such as the sample size and sensitivity of the observer ratings, are also discussed.

Takeover Requests in Highly Automated Truck Driving: How Do the Amount and Type of Additional Information Influence the Driver–Automation Interaction?

Vehicle automation is linked to various benefits, such as increase in fuel and transport efficiency as well as increase in driving comfort. However, automation also comes with a variety of possible downsides, e.g., loss of situational awareness, loss of skills, and inappropriate trust levels regarding system functionality. Drawbacks differ at different automation levels. As highly automated driving (HAD, level 3) requires the driver to take over the driving task in critical situations within a limited period of time, the need for an appropriate human–machine interface (HMI) arises. To foster adequate and efficient human–machine interaction, this contribution presents a user-centered, iterative approach for HMI evaluation of highly automated truck driving. For HMI evaluation, a driving simulator study [n = 32] using a dynamic truck driving simulator was conducted to let users experience the HMI in a semi-real driving context. Participants rated three HMI concepts, differing in their informational content for HAD regarding acceptance, workload, user experience, and controllability. Results showed that all three HMI concepts achieved good to very good results in these measures. Overall, HMI concepts offering more information to the driver about the HAD system showed significantly higher ratings, depicting the positive effect of additional information on the driver–automation interaction.

Effects of Non-Driving Related Task Modalities on Takeover Performance in Highly Automated Driving

OBJECTIVE

Aim of the study was to evaluate the impact of different non-driving related tasks (NDR tasks) on takeover performance in highly automated driving.

BACKGROUND

During highly automated driving, it is allowed to engage in NDR tasks temporarily. However, drivers must be able to take over control when reaching a system limit. There is evidence that the type of NDR task has an impact on takeover performance, but little is known about the specific task characteristics that account for performance decrements.

METHOD

Thirty participants drove in a simulator using a highly automated driving system. Each participant faced five critical takeover situations. Based on assumptions of Wickens's multiple resource theory, stimulus and response modalities of a prototypical NDR task were systematically manipulated. Additionally, in one experimental group, the task was locked out simultaneously with the takeover request.

RESULTS

Task modalities had significant effects on several measures of takeover performance. A visual-manual texting task degraded performance the most, particularly when performed handheld. In contrast, takeover performance with an auditory-vocal task was comparable to a baseline without any task. Task lockout was associated with faster hands-on-wheel times but not altered brake response times.

CONCLUSION

Results showed that NDR task modalities are relevant factors for takeover performance. An NDR task lockout was highly accepted by the drivers and showed moderate benefits for the first takeover reaction.

APPLICATION

Knowledge about the impact of NDR task characteristics is an enabler for adaptive takeover concepts. In addition, it might help regulators to make decisions on allowed NDR tasks during automated driving.

Modeling take-over performance in level 3 conditionally automated vehicles

Taking over vehicle control from a Level 3 conditionally automated vehicle can be a demanding task for a driver. The take-over determines the controllability of automated vehicle functions and thereby also traffic safety. This paper presents models predicting the main take-over performance variables take-over time, minimum time-to-collision, brake application and crash probability. These variables are considered in relation to the situational and driver-related factors time-budget, traffic density, non-driving-related task, repetition, the current lane and driver's age. Regression models were developed using 753 take-over situations recorded in a series of driving simulator experiments. The models were validated with data from five other driving simulator experiments of mostly unrelated authors with another 729 take-over situations. The models accurately captured take-over time, time-to-collision and crash probability, and moderately predicted the brake application. Especially the time-budget, traffic density and the repetition strongly influenced the take-over performance, while the non-driving-related tasks, the lane and drivers' age explained a minor portion of the variance in the take-over performances.

The effect of varying levels of vehicle automation on drivers' lane changing behaviour

Much of the Human Factors research into vehicle automation has focused on driver responses to critical scenarios where a crash might occur. However, there is less knowledge about the effects of vehicle automation on drivers' behaviour during non-critical take-over situations, such as driver-initiated lane-changing or overtaking. The current driving simulator study, conducted as part of the EC-funded AdaptIVe project, addresses this issue. It uses a within-subjects design to compare drivers' lane-changing behaviour in conventional manual driving, partially automated driving (PAD) and conditionally automated driving (CAD). In PAD, drivers were required to re-take control from an automated driving system in order to overtake a slow moving vehicle, while in CAD, the driver used the indicator lever to initiate a system-performed overtaking manoeuvre. Results showed that while drivers' acceptance of both the PAD and CAD systems was high, they generally preferred CAD. A comparison of overtaking positions showed that drivers initiated overtaking manoeuvres slightly later in PAD than in manual driving or CAD. In addition, when compared to conventional driving, drivers had higher deviations in lane positioning and speed, along with higher lateral accelerations during lane changes following PAD. These results indicate that even in situations which are not time-critical, drivers' vehicle control after automation is degraded compared to conventional driving.

Driver observation and shared vehicle control: supporting the driver on the way back into the control loop

In the near future, drivers of automated cars will still have to take over from time to time at short notice. Current control systems implement a hard switch, disabling the automation all at once. However, studies show that the driver’s ability to take over depends on his last activity. We therefore propose a system that uses camera based observation of the driver to assess the situation and to predict transition times. We combine this with a control system that uses a cooperative shared control method to support the driver in takeover situations and allows him to adjust safely to the current situation. We present our first steps towards this goal and show both how the behavior of the driver in the interior can be assessed and how a cooperative control transfer can be implemented. We further point out the necessary steps to implement the proposed system and give a first impression of the performance via simulation.

How we can measure the non-driving-task engagement in automated driving: Comparing flow experience and workload

In automated driving, a driver can completely concentrate on non-driving-related tasks (NDRTs). This study investigated the flow experience of a driver who concentrated on NDRTs and tasks that induce mental workload under conditional automation. Participants performed NDRTs under different demand levels: a balanced demand-skill level (fit condition) to induce flow, low-demand level to induce boredom, and high-demand level to induce anxiety. In addition, they performed the additional N-Back task, which artificially induces mental workload. The results showed participants had the longest reaction time when they indicated the highest flow score, and had the longest gaze-on time, road-fixation time, hands-on time, and take-over time under the fit condition. Significant differences were not observed in the driver reaction times in the fit condition and the additional N-Back task, indicating that performing NDRTs that induce a high flow experience could influence driver reaction time similar to performing tasks with a high mental workload.

Introduction matters: Manipulating trust in automation and reliance in automated driving

Trust in automation is a key determinant for the adoption of automated systems and their appropriate use. Therefore, it constitutes an essential research area for the introduction of automated vehicles to road traffic. In this study, we investigated the influence of trust promoting (Trust promoted group) and trust lowering (Trust lowered group) introductory information on reported trust, reliance behavior and take-over performance. Forty participants encountered three situations in a 17-min highway drive in a conditionally automated vehicle (SAE Level 3). Situation 1 and Situation 3 were non-critical situations where a take-over was optional. Situation 2 represented a critical situation where a take-over was necessary to avoid a collision. A non-driving-related task (NDRT) was presented between the situations to record the allocation of visual attention. Participants reporting a higher trust level spent less time looking at the road or instrument cluster and more time looking at the NDRT. The manipulation of introductory information resulted in medium differences in reported trust and influenced participants' reliance behavior. Participants of the Trust promoted group looked less at the road or instrument cluster and more at the NDRT. The odds of participants of the Trust promoted group to overrule the automated driving system in the non-critical situations were 3.65 times (Situation 1) to 5 times (Situation 3) higher. In Situation 2, the Trust promoted group's mean take-over time was extended by 1154 ms and the mean minimum time-to-collision was 933 ms shorter. Six participants from the Trust promoted group compared to no participant of the Trust lowered group collided with the obstacle. The results demonstrate that the individual trust level influences how much drivers monitor the environment while performing an NDRT. Introductory information influences this trust level, reliance on an automated driving system, and if a critical take-over situation can be successfully solved.