Automation Expectation Mismatch: Incorrect Prediction Despite Eyes on Threat and Hands on Wheel

Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood?

The present study investigated the effects of a driver monitoring system that triggers attention warnings in case distraction is detected. Based on the EuroNCAP protocol, distraction could either be long glances away from the forward roadway (≥3s) or visual attention time sharing (>10 cumulative seconds within a 30 s time interval). In a series of manual driving simulator drives, 30 participants completed both driving related tasks (e.g., changing multiple lanes in dense traffic) and non-driving related tasks (e.g., infotainment operations). Results of warning frequencies revealed that visual attention time sharing warnings occurred more frequently than long distraction warnings. Moreover, there was a large number of attention warnings during driving related tasks. Results also revealed that participants' mental models tended to be less accurate when it came to understanding of the visual attention time sharing warnings as compared to the long distraction warnings, which were understood more accurately. Based on these observations, the work discusses the applicability and design of driver monitoring warnings.

Test procedure for the evaluation of partially automated driving HMI including driver monitoring systems in driving simulation

The proposed test procedure presents an approach for the evaluation of the usability of partial automated driving HMI including driver monitoring systems in driving simulation. This procedure is based on a definition of requirements that a Level 2 HMI and its included driver monitoring system must fulfill in order to guarantee that the drivers understand their responsibilities of continuously monitoring the driving environment and the status of the partial automated driving system. These requirements are used to define the evaluation criteria that have to be validated in the test as well as the use cases in which these criteria can be assessed. The result is a detailed and comprehensive test guide including the specification of the test drives, the necessary instructions, the test environment and the recruiting criteria for the test sample.•Evaluation of usability aspects of level 2 automated driving HMI including driver monitoring systems•Based on the definition of requirements for L2 HMI•Test guide including the definition of use cases, evaluation criteria and testing conditions in driving simulation.

Driver Visual Attention Before and After Take-Over Requests During Automated Driving on Public Roads

OBJECTIVE

This study aims to understand drivers' visual attention before and after take-over requests during automated driving (AD), when the vehicle is fully responsible for the driving task on public roads.

BACKGROUND

Existing research on transitions of control from AD to manual driving has mainly focused on take-over times. Despite its relevance for vehicle safety, drivers' visual attention has received little consideration.

METHOD

Thirty participants took part in a Wizard of Oz study on public roads. Drivers' visual attention was analyzed before and after four take-over requests. Visual attention during manual driving was also recorded to serve as a baseline for comparison.

RESULTS

During AD, the participants showed reduced visual attention to the forward road and increased duration of single off-road glances compared to manual driving. In response to take-over requests, the participants looked away from the forward road toward the instrument cluster. Levels of visual attention towards the forward road did not return to the levels observed during manual driving until after 15 s had passed.

CONCLUSION

During AD, drivers may look toward non-driving related task items (e.g., mobile phone) instead of forward. Further, when a transition of control is required, drivers may take over control before they are aware of the driving environment or potential threat(s). Thus, it cannot be assumed that drivers are ready to respond to events shortly after the take-over request.

APPLICATION

It is important to consider the effect of the design of take-over requests on drivers' visual attention alongside take-over times.

Habits, attitudes, and expectations of regular users of partial driving automation systems

INTRODUCTION

Little is known about regular users' perceptions of partial (Level 2) automation or how those perceptions affect behind-the-wheel behavior.

METHOD

A mixed mode (phone and online) survey explored the habits, expectations, and attitudes among regular users of General Motors Super Cruise (n = 200), Nissan/Infiniti ProPILOT Assist (n = 202), and Tesla Autopilot (n = 202).

RESULTS

All three groups reported being more likely to engage in non-driving-related activities while using their systems than while driving unassisted. Super Cruise and Autopilot users especially were more likely to report engaging in activities that involved taking their hands off the wheel or their eyes off the road. Many Super Cruise and Autopilot users also said they could perform secondary (non-driving-related) tasks better and more often while using their systems, while fewer ProPILOT Assist users shared this opinion. Super Cruise users were most likely and ProPILOT Assist users least likely to think that secondary activities were safer to perform while using their systems. While some drivers said they found user safeguards (e.g., attention reminders, lockouts) annoying and tried to circumvent them, most people said they found them helpful and felt safer with them. Large percentages of users (53% Super Cruise, 42% Autopilot and 12% ProPILOT Assist) indicated they were comfortable treating their systems as self-driving.

CONCLUSIONS

Some regular users have a poor understanding of their technology's limits. System design appears to contribute to user perceptions and behavior. However, owner populations also differ, which means habits, attitudes, and expectations may not generalize. Most people value user safeguards, but some implementations may not be effective for everyone.

PRACTICAL APPLICATIONS

Multifaceted, proactive user-centric safeguards are needed to shape proper behavior and understanding about drivers' roles and responsibilities while using partial driving automation.

Drivers' Performance in Non-critical Take-Overs From an Automated Driving System-An On-Road Study

OBJECTIVE

The objective of this semi-controlled study was to investigate drivers' performance when resuming control from an Automated Driving System (ADS), simulated through the Wizard of Oz method, in real traffic.

BACKGROUND

Research on take-overs has primarily focused on urgent scenarios. This article aims to shift the focus to non-critical take-overs from a system operating in congested traffic situations.

METHOD

Twenty drivers drove a selected route in rush-hour traffic in the San Francisco Bay Area, CA, USA. During the drive, the ADS became available when predetermined availability conditions were fulfilled. When the system was active, the drivers were free to engage in non-driving related activities.

RESULTS

The results show that drivers' transition time goes down with exposure, making it reasonable to assume that some experience is required to regain control with comfort and ease. The novel analysis of after-effects of automated driving on manual driving performance implies that the after-effects were close to negligible. Observational data indicate that, with exposure, a majority of the participants started to engage in non-driving related activities to some extent, but it is unclear how the activities influenced the take-over performance.

CONCLUSION

The results indicate that drivers need repeated exposure to take-overs to be able to fully resume manual control with ease.

APPLICATION

Take-over signals (e.g., visuals, sounds, and haptics) should be carefully designed to avoid startle effects and the human-machine interface should provide clear guidance on the required take-over actions.

Evaluation of Imminent Take-Over Requests With Real Automation on a Test Track

OBJECTIVE

Investigating take-over, driving, non-driving related task (NDRT) performance, and trust of conditionally automated vehicles (AVs) in critical transitions on a test track.

BACKGROUND

Most experimental results addressing driver take-over were obtained in simulators. The presented experiment aimed at validating relevant findings while uncovering potential effects of motion cues and real risk.

METHOD

Twenty-two participants responded to four critical transitions on a test track. Non-driving related task modality (reading on a handheld device vs. auditory) and take-over timing (cognitive load) were varied on two levels. We evaluated take-over and NDRT performance as well as gaze behavior. Further, trust and workload were assessed with scales and interviews.

RESULTS

Reaction times were significantly faster than in simulator studies. Further, reaction times were only barely affected by varying visual, physical, or cognitive load. Post-take-over control was significantly degraded with the handheld device. Experiencing the system reduced participants' distrust, and distrusting participants monitored the system longer and more frequently. NDRTs on a handheld device resulted in more safety-critical situations.

CONCLUSION

The results confirm that take-over performance is mainly influenced by visual-cognitive load, while physical load did not significantly affect responses. Future take-over request (TOR) studies may investigate situation awareness and post-take-over control rather than reaction times only. Trust and distrust can be considered as different dimensions in AV research.

APPLICATION

Conditionally AVs should offer dedicated interfaces for NDRTs to provide an alternative to using nomadic devices. These interfaces should be designed in a way to maintain drivers' situation awareness.

PRÉCIS

This paper presents a test track experiment addressing conditionally automated driving systems. Twenty-two participants responded to critical TORs, where we varied NDRT modality and take-over timing. In addition, we assessed trust and workload with standardized scales and interviews.

Factors Influencing Trust in Advanced Driver Assistance Systems for Current Users

Understanding the factors influencing trust in advanced driver assistance systems (ADAS) may help inform training and education to support appropriate use. We surveyed 369 drivers with experience using both adaptive cruise control (ACC) and lane keeping assist (LKA). The survey included questions to assess trust in ADAS, along with objective knowledge about ADAS limitations, self-reported understanding of ADAS, familiarity with technology, propensity to trust technology, and demographics. Regression results showed that self-reported understanding, but not objective knowledge, predicted trust in ADAS. Self-reported understanding was not correlated with objective knowledge; overall, participants were not aware of many of the system limitations included in the survey. Propensity to trust technology was also a significant predictor of trust. Training/educational materials could be designed to inform drivers of potential gaps in their understanding and adjust expectations of ADAS to support appropriate trust for those with a high propensity to trust technology.

Near-Perfect Automation: Investigating Performance, Trust, and Visual Attention Allocation

OBJECTIVE

Assess performance, trust, and visual attention during the monitoring of a near-perfect automated system.

BACKGROUND

Research rarely attempts to assess performance, trust, and visual attention in near-perfect automated systems even though they will be relied on in high-stakes environments.

METHODS

Seventy-three participants completed a 40-min supervisory control task where they monitored three search feeds. All search feeds were 100% reliable with the exception of two automation failures: one miss and one false alarm. Eye-tracking and subjective trust data were collected.

RESULTS

Thirty-four percent of participants correctly identified the automation miss, and 67% correctly identified the automation false alarm. Subjective trust increased when participants did not detect the automation failures and decreased when they did. Participants who detected the false alarm had a more complex scan pattern in the 2 min centered around the automation failure compared with those who did not. Additionally, those who detected the failures had longer dwell times in and transitioned to the center sensor feed significantly more often.

CONCLUSION

Not only does this work highlight the limitations of the human when monitoring near-perfect automated systems, it begins to quantify the subjective experience and attentional cost of the human. It further emphasizes the need to (1) reevaluate the role of the operator in future high-stakes environments and (2) understand the human on an individual level and actively design for the given individual when working with near-perfect automated systems.

APPLICATION

Multiple operator-level measures should be collected in real-time in order to monitor an operator's state and leverage real-time, individualized assistance.

Do you trust me? Driver responses to automated evasive maneuvers

An increasing number of Conditionally Automated Driving (CAD) systems are being developed by major automotive manufacturers. In a CAD system, the automated system is in control of the vehicle within its operational design domain. Therefore, in CAD the vehicle is capable of tactical control of the vehicle and needs to be able to maneuver evasively by braking or steering to avoid objects. During these evasive maneuvers, the driver may attempt to take back control of the vehicle by intervening. A driver interrupting a CAD vehicle while properly performing an evasive maneuver presents a potential safety risk. To investigate this issue, 36 participants were recruited to participate in a Wizard-of-Oz research study. The participants experienced one of two evasive maneuvers of moderate intensity on a test track. The evasive maneuver required the CAD system to brake or steer to avoid the box placed in the lane of travel of the test vehicle. Drivers glanced toward the obstacle but did not intervene or prepare to intervene in response to the evasive maneuver. Importantly, the drivers who chose to intervene did so safely. These findings suggest that after experiencing a CAD vehicle for a brief period, most participants trusted the system enough to not intervene during a system-initiated evasive maneuver.

The effect of a startle-based warning, age, sex, and secondary task on takeover actions in critical autonomous driving scenarios

Introduction: In highly autonomous driving scenarios, it is critical to identify strategies to accelerate reaction times since drivers may take too long to take over control of the vehicle. Previous studies reported that an Acoustic Startling Pre-Stimulus (ASPS, i.e., a loud warning preceding an action) accelerated reaction times in simple ankle flexion exercises.

Methods: In this study, we examined if an ASPS warning leads to shorter takeover reaction times in a sled-simulated evasive swerving maneuver. Twenty-eight participants (seven male adults, seven male teenagers, seven female adults, and seven female teenagers) were instructed to align a marker on the steering wheel with a marker on a lateral post as fast as they could as soon as the lateral sled perturbation (0.75 g) started. Four conditions were examined: with and without an ASPS (105 dB, played 250 ms before sled perturbation for 40 ms), and with and without a secondary task (i.e., texting). A catch trial (ASPS only) was used to minimize anticipation. Human kinematics were captured with an 8-camera 3D motion capture system.

Results: Results showed that the drivers’ hands lifted towards the steering wheel more quickly with the ASPS (169 ± 55 ms) than without (194 ± 46 ms; p = 0.01), and that adult drivers touched the steering wheel quicker with the ASPS (435 ± 54 ms) than without (470 ± 33 ms; p = 0.01). Similar findings were not observed for the teen drivers. Additionally, female drivers were found to lift their hands towards the steering wheel faster than male drivers (166 ± 58 ms vs. 199 ± 36 ms; p = 0.009).

Discussion: Our findings suggest that the ASPS may be beneficial to accelerate driver reaction times during the initiation of a correction maneuver, and that autonomous vehicle warnings may need to be tailored to the age and sex of the driver.

A Bayesian Regression Analysis of the Effects of Alert Presence and Scenario Criticality on Automated Vehicle Takeover Performance

OBJECTIVE

This study investigates the impact of silent and alerted failures on driver performance across two levels of scenario criticality during automated vehicle transitions of control.

BACKGROUND

Recent analyses of automated vehicle crashes show that many crashes occur after a transition of control or a silent automation failure. A substantial amount of research has been dedicated to investigating the impact of various factors on drivers' responses, but silent failures and their interactions with scenario criticality are understudied.

METHOD

A driving simulator study was conducted comparing scenario criticality, alert presence, and two driving scenarios. Bayesian regression models and Fisher's exact tests were used to investigate the impact of alert and scenario criticality on takeover performance.

RESULTS

The results show that silent failures increase takeover times and the intensity of posttakeover maximum accelerations and decrease the posttakeover minimum time-to-collision. While the predicted average impact of silent failures on takeover time was practically low, the effects on minimum time-to-collision and maximum accelerations were safety-significant. The analysis of posttakeover control interaction effects shows that the effect of alert presence differs by the scenario criticality.

CONCLUSION

Although the impact of the absence of an alert on takeover performance was less than that of scenario criticality, silent failures seem to play a substantial role-by leading to an unsafe maneuver-in critical automated vehicle takeovers.

APPLICATION

Understanding the implications of silent failure on driver's takeover performance can benefit the assessment of automated vehicles' safety and provide guidance for fail-safe system designs.

Driving Activity Recognition Using UWB Radar and Deep Neural Networks

In-car activity monitoring is a key enabler of various automotive safety functions. Existing approaches are largely based on vision systems. Radar, however, can provide a low-cost, privacy-preserving alternative. To this day, such systems based on the radar are not widely researched. In our work, we introduce a novel approach that uses the Doppler signal of an ultra-wideband (UWB) radar as an input to deep neural networks for the classification of driving activities. In contrast to previous work in the domain, we focus on generalization to unseen persons and make a new radar driving activity dataset (RaDA) available to the scientific community to encourage comparison and the benchmarking of future methods.

Driver response and recovery following automation initiated disengagement in real-world hands-free driving

OBJECTIVE

Advanced driver assistance systems are increasingly available in consumer vehicles, making the study of drivers' behavioral adaptation and the impact of automation beneficial for driving safety. Concerns over driver's being out-of-the-loop, coupled with known limitations of automation, has led research to focus on time-critical, system-initiated disengagements. This study used real-world data to assess drivers' response to, and recovery from, automation-initiated disengagements by quantifying changes in visual attention, vehicle control, and time to steady-state behaviors.

METHODS

Fourteen drivers drove for one month each a Cadillac CT6 equipped with Super Cruise (SC), a partial automation system that, when engaged, enables hands-free driving. The vehicles were instrumented with data acquisition systems recording driving kinematics, automation use, GPS, and video. The dataset included 265 SC-initiated disengagements identified across 5,514 miles driven with SC.

RESULTS

Linear quantile mixed-effects models of glance behavior indicated that following SC-initiated disengagement, the proportions of glances to the Road decreased (Q50_Before=0.91, Q50_After=0.69; Q85_Before=1.0, Q85_After=0.79), the proportions of glances to the Instrument Cluster increased (Q50_Before=0.14, Q50_After=0.25; Q85_Before=0.34, Q85_After=0.45), and mean glance duration to the Road decreased by 4.86 sec in Q85. Multinomial logistic regression mixed-models of glance distributions indicated that the number of transitions between glance locations following disengagement increased by 43% and that glances were distributed across fewer locations. When driving hands-free, take over time was significantly longer (2.4 sec) compared to when driving with at least one hand on the steering wheel (1.8 sec). Analysis of moment-to-moment distributional properties of visual attention and steering wheel control following disengagement indicated that on average it took drivers 6.1 sec to start the recovery of glance behavior to the Road and 1.5 sec for trend-stationary proportions of at least one hand on the steering wheel.

CONCLUSIONS

Automation-initiated disengagements triggered substantial changes in driver glance behavior including shorter on-road glances and frequent transitions between Road and Instrument Cluster glance locations. This information seeking behavior may capture drivers' search for information related to the disengagement or the automation state and is likely shaped by the automation design. The study findings can inform the design of more effective driver-centric information displays for smoother transitions and faster recovery.

Drivers don't need to learn all ADAS limitations: A comparison of limitation-focused and responsibility-focused training approaches

Expecting drivers to learn and remember numerous limitations may not be a practical approach to training for advanced driver assistance systems (ADAS), particularly for self-initiated training in the absence of formal training requirements. One alternative is focusing on the importance of the driver remaining engaged in the driving task (responsibility-focused approach). We investigated the effects of two training videos (responsibility-focused and limitation-focused) on reliance intention, trust, and ADAS knowledge. In a remote study, participants (N = 61) watched dashcam clips (8 that require takeover, 8 no takeover) and for each clip, they reported whether they would manually intervene and their trust in ADAS (assessing situational reliance intention and trust, respectively). Participants also completed a questionnaire that included items measuring ADAS knowledge. Responses were collected at three stages: pre-training, post-training, and a follow-up session (minimum four weeks later). There were no significant differences between approaches in terms of knowledge of situations in which ADAS would not work, appropriate situational reliance intention, or trust in takeover scenarios. Compared to the responsibility-focused video, the limitation-focused video was associated with lower trust in no takeover scenarios and negative bias at post-training (i.e., bias towards reporting that ADAS would not work for the knowledge questionnaire and bias towards taking manual control/not using ADAS for the dashcam clips). Given the limited differences between training approaches and potential drawbacks of the limitation-focused approach, our results suggest that the responsibility-focused training approach is worth exploring further.

Scan-based eye tracking measures are predictive of workload transition performance

Given there is no unifying theory or design guidance for workload transitions, this work investigated how visual attention allocation patterns could inform both topics, by understanding if scan-based eye tracking metrics could predict workload transition performance trends in a context-relevant domain. The eye movements of sixty Naval flight students were tracked as workload transitioned at a slow, medium, and fast pace in an unmanned aerial vehicle testbed. Four scan-based metrics were significant predictors across the different growth curve models of response time and accuracy. Stationary gaze entropy (a measure of how dispersed visual attention transitions are across tasks) was predictive across all three transition rates. The other three predictive scan-based metrics captured different aspects of visual attention, including its spread, directness, and duration. The findings specify several missing details in both theory and design guidance, which is unprecedented, and serves as a basis of future workload transition research.

What is new with Artificial Intelligence? Human–agent interactions through the lens of social agency

In this article, we suggest that the study of social interactions and the development of a “sense of agency” in joint action can help determine the content of relevant explanations to be implemented in artificial systems to make them “explainable.” The introduction of automated systems, and more broadly of Artificial Intelligence (AI), into many domains has profoundly changed the nature of human activity, as well as the subjective experience that agents have of their own actions and their consequences – an experience that is commonly referred to as sense of agency. We propose to examine the empirical evidence supporting this impact of automation on individuals’ sense of agency, and hence on measures as diverse as operator performance, system explicability and acceptability. Because of some of its key characteristics, AI occupies a special status in the artificial systems landscape. We suggest that this status prompts us to reconsider human–AI interactions in the light of human–human relations. We approach the study of joint actions in human social interactions to deduce what key features are necessary for the development of a reliable sense of agency in a social context and suggest that such framework can help define what constitutes a good explanation. Finally, we propose possible directions to improve human–AI interactions and, in particular, to restore the sense of agency of human operators, improve their confidence in the decisions made by artificial agents, and increase the acceptability of such agents.

How to Keep Drivers Attentive during Level 2 Automation? Development and Evaluation of an HMI Concept Using Affective Elements and Message Framing

With Level 3 and 4 automated driving activated, users will be allowed to engage in a wide range of non-driving related activities (NDRAs). Although Level 2 automation can appear very similar to L3 and L4, drivers are required to always monitor the system. However, past research has found drivers neglect this obligation at least partly and instead engage in NDRAs. Since this behavior can have negative impacts on traffic safety, the goal of this work was to develop a human–machine interface (HMI) concept to motivate users to continue their supervision task. This work’s concept used message framing in connection with affective elements. Every three minutes, messages were displayed on the head-up display. To evaluate the affective message concept’s (AMC) effectiveness, we conducted a between-subject driving simulator study (baseline vs. advanced HMI) with 32 participants and 45 min of driving time with both L2 and L4 phases and a silent system malfunction. Results show the road attention ratio decreases and the NDRA engagement ratio increases over time only for baseline participants. Participants supported by the AMC did not show a change over time in monitoring behavior and NDRA engagement. However, no effect on the drivers’ reaction to the system failure became apparent. No effects on subjective workload and user experience were found. Additional research is needed to further investigate the safety implications and long-term effectiveness of the concept, as well as a driver-state-dependent design.

Objective assessment of trait attentional control predicts driver response to emergency failures of vehicular automation

With the advent of autonomous driving, the issue of human intervention during safety-critical events is an urgent topic of research. Supervisory monitoring, taking over vehicle control during automation failures and then bringing the vehicle to safety under time pressure are cognitively demanding tasks that pose varying difficulties across the driving population. This underpins a need to investigate individual differences (i.e., how people differ in their dispositional traits) in driver responses to automation system limits, so that autonomous vehicle design can be tailored to meet the safety-critical needs of higher-risk drivers. However, few studies thus far have examined individual differences, with self-report measures showing limited ability to predict driver takeover performance. To address this gap, the present study explored the utility of an established brain activity-based objective index of trait attentional control (frontal theta/beta ratio; TBR) in predicting driver interactions with conditional automation. Frontal TBR predicted drivers' average takeover reaction time, as well as the likelihood of accident after takeover. Moving towards practical applications, this study also demonstrated the utility of streamlined estimates of frontal TBR measured from the forehead electrodes and from a single crown electrode, with the latter showing better fidelity and predictive value. Overall, TBR is behaviourally relevant, measurable with minimal sensors and easily computable, rendering it a promising candidate for practical and objective assessment of drivers' neurocognitive traits that contribute to their AV driving readiness.

Effects of Distraction in On-Road Level 2 Automated Driving: Impacts on Glance Behavior and Takeover Performance

OBJECTIVE

The paper aimed to investigate glance behaviors under different levels of distraction in automated driving (AD) and understand the impact of distraction levels on driver takeover performance.

BACKGROUND

Driver distraction detrimentally affects takeover performance. Glance-based distraction measurement could be a promising method to remind drivers to maintain enough attentiveness before the takeover request in partially AD.

METHOD

Thirty-six participants were recruited to drive a Tesla Model S in manual and Autopilot modes on a test track while engaging in secondary tasks, including temperature-control, email-sorting, and music-selection, to impose low and high distractions. During the test drive, participants needed to quickly change the lane as if avoiding an immediate road hazard if they heard an unexpected takeover request (an auditory warning). Driver state and behavior over the test drive were recorded in real time by a driver monitoring system and several other sensors installed in the Tesla vehicle.

RESULTS

The distribution of off-road glance duration was heavily skewed (with a long tail) by high distractions, with extreme glance duration more than 30 s. Moreover, being eyes-off-road before takeover could cause more delay in the urgent takeover reaction compared to being hands-off-wheel.

CONCLUSION

The study measured off-road glance duration under different levels of distraction and demonstrated the impacts of being eyes-off-road and hands-off-wheel on the following takeover performance.

APPLICATION

The findings provide new insights about engagement in Level 2 AD and are useful for the design of driver monitoring technologies for distraction management.

Keeping the driver in the loop through semi-automated or manual lane changes in conditionally automated driving

In the current study we investigated if drivers of conditionally automated vehicles can be kept in the loop through lane change maneuvers. More specifically, we examined whether involving drivers in lane-changes during a conditionally automated ride can influence critical take-over behavior and keep drivers' gaze on the road. In a repeated measures driving simulator study (n = 85), drivers drove the same route three times, each trial containing four lane changes that were all either (1) automated, (2) semi-automated or (3) manual. Each ride ended with a critical take-over situation that could be solved by braking and/or steering. Critical take-over reactions were analyzed with a linear mixed model and parametric accelerated failure time survival analysis. As expected, semi-automated and manual lane changes throughout the ride led to 13.5% and 17.0% faster maximum deceleration compared to automated lane changes. Additionally, semi-automated and manual lane changes improved the quality of the take-over by significantly decreasing standard deviation of the steering wheel angle. Unexpectedly, drivers in the semi-automated condition were slowest to start the braking maneuver. This may have been caused by the drivers' confusion as to how the semi-automated system would react. Additionally, the percentage gaze off-the-road was significantly decreased by the semi-automated (6.0%) and manual (6.6%) lane changes. Taken together, the results suggest that semi-automated and manual transitions may be an alarm-free instrument which developers could use to help maintain drivers' perception-action loop and improve automated driving safety.

A model for naturalistic glance behavior around Tesla Autopilot disengagements

OBJECTIVE

We present a model for visual behavior that can simulate the glance pattern observed around driver-initiated, non-critical disengagements of Tesla's Autopilot (AP) in naturalistic highway driving.

BACKGROUND

Drivers may become inattentive when using partially-automated driving systems. The safety effects associated with inattention are unknown until we have a quantitative reference on how visual behavior changes with automation.

METHODS

The model is based on glance data from 290 human initiated AP disengagement epochs. Glance duration and transition were modelled with Bayesian Generalized Linear Mixed models.

RESULTS

The model replicates the observed glance pattern across drivers. The model's components show that off-road glances were longer with AP active than without and that their frequency characteristics changed. Driving-related off-road glances were less frequent with AP active than in manual driving, while non-driving related glances to the down/center-stack areas were the most frequent and the longest (22% of the glances exceeded 2 s). Little difference was found in on-road glance duration.

CONCLUSION

Visual behavior patterns change before and after AP disengagement. Before disengagement, drivers looked less on road and focused more on non-driving related areas compared to after the transition to manual driving. The higher proportion of off-road glances before disengagement to manual driving were not compensated by longer glances ahead.

APPLICATION

The model can be used as a reference for safety assessment or to formulate design targets for driver management systems.

Why human factors science is demonstrably necessary: historical and evolutionary foundations

We review the theoretical foundation for the need for human factors science. Over the past 2.8 million years, humans and tools have co-evolved. However, in the last century, technology is introduced at a rate that exceeds human evolution. The proliferation of computers and, more recently, robots, introduces new cognitive demands, as the human is required to be a monitor rather than a direct controller. The usage of robots and artificial intelligence is only expected to increase, and the present COVID-19 pandemic may prove to be catalytic in this regard. One way to improve overall system performance is to 'adapt the human to the machine' via task procedures, operator training, operator selection, a Procrustean mandate. Using classic research examples, we demonstrate that Procrustean methods can improve performance only to a limited extent. For a viable future, therefore, technology must adapt to the human, which underwrites the necessity of human factors science. Practitioner Summary: Various research articles have reported that the science of Human Factors is of vital importance in improving human-machine systems. However, what is lacking is a fundamental historical outline of why Human Factors is important. This article provides such a foundation, using arguments ranging from pre-history to post-COVID.

Tools for Transport: Driven to Learn With Connected Vehicles

Vehicle automation and assistance technologies have been touted as a means to reduce traffic collisions by minimizing or eliminating "error-prone" and inefficient human operators. In concept, automation exists on a continuum that includes engaged driving by a human operator augmented by automated support features, vigilant driver monitoring of vehicle behavior with the possibility of driver take-over, to full automation with no active monitoring by a human operator. Moreover, the degree of automation varies by vehicle features (e.g., lane centering, emergency braking, adaptive cruise control, parking), by setting, meaning that automated features may or may not be available depending on specific attributes of the traffic environment (e.g., traffic volume, road geometry, etc), and by implementation (e.g., haptic vs. auditory warnings). Thus, these automotive "transportation tools" are highly heterogeneous and pose unique challenges and opportunities for driver training. In this paper, we report the results of an experimental study (n = 36) to determine if enhanced vehicle feedback influences driver trust, effort, frustration, and performance (indexed by reaction time) in a virtual driving environment. Results are contextualized in the extant literature on learning to operate motor vehicles and outline key research questions essential for understanding the processes by which skilled performance develops with respect to a real-world practical tool: the increasingly automated automobile.

Evaluating feedback requirements for trust calibration in automated vehicles

The inappropriate use of automation as a result of trust issues is a major barrier for a broad market penetration of automated vehicles. Studies so far have shown that providing information about the vehicle’s actions and intentions can be used to calibrate trust and promote user acceptance. However, how such feedback could be designed optimally is still an open question. This article presents the results of two user studies. In the first study, we investigated subjective trust and user experience of (N=21) participants driving in a fully automated vehicle, which interacts with other traffic participants in virtual reality. The analysis of questionnaires and semi-structured interviews shows that participants request feedback about the vehicle’s status and intentions and prefer visual feedback over other modalities. Consequently, we conducted a second study to derive concrete requirements for future feedback systems. We showed (N=56) participants various videos of an automated vehicle from the ego perspective and asked them to select elements in the environment they want feedback about so that they would feel safe, trust the vehicle, and understand its actions. The results confirm a correlation between subjective user trust and feedback needs and highlight essential requirements for automatic feedback generation. The results of both experiments provide a scientific basis for designing more adaptive and personalized in-vehicle interfaces for automated driving.

Deceleration Assistance Mitigated the Trade-off Between Sense of Agency and Driving Performance

Driving assistance technology has gained traction in recent years and is becoming more widely used in vehicles. However, drivers usually experience a reduced sense of agency when driving assistance is active even though automated assistance improves driving performance by reducing human error and ensuring quick reactions. The present study examined whether driving assistance can maintain human sense of agency during early deceleration in the face of collision risk, compared with manual deceleration. In the experimental task, participants decelerate their vehicle in a driving simulator to avoid collision with a vehicle that suddenly cut in front of them and decelerated. In the assisted condition, the system performed deceleration 100 ms after the cut-in. Participants were instructed to decelerate their vehicle and follow the vehicle that cut-in. This design ensured that the deceleration assistance applied a similar control to the vehicle as the drivers intended to, only faster and smoother. Participants rated their sense of agency and their driving performance. The results showed that drivers maintained their sense of agency and improved driving performance under driving assistance. The findings provided insights into designing driving assistance that can maintain drivers' sense of agency while improving future driving performance. It is important to establish a mode of joint-control in which the system shares the intention of human drivers and provides improved execution of control.

Knowledge of and trust in advanced driver assistance systems

Understanding what drivers know about state-of-the-art advanced driver assistance systems (ADAS), like adaptive cruise control (ACC) and lane keeping assistance (LKA) is important because such knowledge can influence trust in and reliance on the automation. We surveyed ADAS owners (N = 102) and non-owners (N = 262), with the primary objective of assessing knowledge and trust of ACC and LKA, and investigating the relationship between knowledge and trust among drivers who have not received special training. The survey contained demographic questions, ACC and LKA knowledge questionnaires (assessing knowledge of capabilities and limitations commonly found in owner's manuals), and ACC and LKA trust ratings. From the knowledge questionnaires, sensitivity (i.e., knowledge of the true capabilities of ACC and LKA) and response bias were assessed and used to predict trust. Results showed that owners did not have better knowledge of system capabilities/limitations than non-owners, in fact, owners had a stronger bias in favour of system capabilities. For non-owners, better knowledge of system capabilities was associated with lower trust, and those who were more biased towards endorsing system capabilities had higher trust. Neither knowledge nor response bias was associated with trust among owners. Further research is needed to confirm our results with a larger sample of owners, but given that it is also impractical to expect drivers to learn and remember all possible ADAS limitations, it may be beneficial to focus training efforts on improving drivers' overall understanding of the fallibility of ADAS and reinforcing their role when using ADAS to support appropriate trust and reliance.

Assessment of Driving Proficiency When Drivers Utilize Assistance Systems—The Case of Adaptive Cruise Control

Driver assistance systems (ADAS), and especially those containing driving automation, change the role of drivers to supervisors who need to safeguard the system’s operation. Despite the aim to increase safety, the new tasks (supervision and intervention) may jeopardize safety. Consequently, safety officers address the need for specific training on ADAS. However, these tasks are not assessed in driver licensing today. Therefore, we developed a framework to assess in-practice driving proficiency when drivers utilize ADAS. This study evaluated whether the proposed framework is able to identify meaningful differences in driving proficiency between driving with and without assistance. We applied the framework to perform a qualitative assessment of driving proficiency with 12 novice drivers in a field experiment, comparable to a license test. The assistance system concerned Adaptive Cruise Control (ACC). The test showed that driving with ACC has a negative influence on self-initiated manoeuvres (especially lane changes) and sometimes led to improved adaptations to manoeuvres initiated by other road users (like merging in traffic). These results are in line with previous research and demonstrate the framework’s successfulness to assess novice drivers’ proficiency to utilize ADAS in road-traffic. Therewith, the proposed framework provides important means for driving instructors and examiners to address the safe operation of ADAS.

Intelligent Badminton Training Robot in Athlete Injury Prevention Under Machine Learning

This study was developed to explore the role of the intelligent badminton training robot (IBTR) to prevent badminton player injuries based on the machine learning algorithm. An IBTR is designed from the perspectives of hardware and software systems, and the movements of the athletes are recognized and analyzed with the hidden Markov model (HMM) under the machine learning. After the design was completed, it was simulated with the computer to analyze its performance. The results show that after the HMM is optimized, the recognition accuracy or data pre-processing algorithm, based on the sliding window segmentation at the moment of hitting reaches 96.03%, and the recognition rate of the improved HMM to the robot can be 94.5%, showing a good recognition effect on the training set samples. In addition, the accuracy rate is basically stable when the total size of the training data is 120 sets, after the accuracy of the robot is analyzed through different data set sizes. Therefore, it was found that the designed IBTR has a high recognition rate and stable accuracy, which can provide experimental references for injury prevention in athlete training.

How Do Drivers Respond to Silent Automation Failures? Driving Simulator Study and Comparison of Computational Driver Braking Models

OBJECTIVE

This paper aims to describe and test novel computational driver models, predicting drivers' brake reaction times (BRTs) to different levels of lead vehicle braking, during driving with cruise control (CC) and during silent failures of adaptive cruise control (ACC).

BACKGROUND

Validated computational models predicting BRTs to silent failures of automation are lacking but are important for assessing the safety benefits of automated driving.

METHOD

Two alternative models of driver response to silent ACC failures are proposed: a looming prediction model, assuming that drivers embody a generative model of ACC, and a lower gain model, assuming that drivers' arousal decreases due to monitoring of the automated system. Predictions of BRTs issued by the models were tested using a driving simulator study.

RESULTS

The driving simulator study confirmed the predictions of the models: (a) BRTs were significantly shorter with an increase in kinematic criticality, both during driving with CC and during driving with ACC; (b) BRTs were significantly delayed when driving with ACC compared with driving with CC. However, the predicted BRTs were longer than the ones observed, entailing a fitting of the models to the data from the study.

CONCLUSION

Both the looming prediction model and the lower gain model predict well the BRTs for the ACC driving condition. However, the looming prediction model has the advantage of being able to predict average BRTs using the exact same parameters as the model fitted to the CC driving data.

APPLICATION

Knowledge resulting from this research can be helpful for assessing the safety benefits of automated driving.

Transport safety and human factors in the era of automation: What can transport modes learn from each other?

One of the main aims of introducing automation in transport is to improve safety by reducing or eliminating human errors; it is often argued however that this may induce new types of errors. There is different level of maturity with automation in different transport modes (road, aviation, maritime and rail), however no systematic research has been conducted on the lessons learned in different sectors, so that they can be exploited for the design of safer automated systems. The aim of this paper is to review the impact of key human factors on the safety of automated transport systems, with focus on relevant experiences from different transport sectors. A systematic literature review is carried out on the following topics: the level of trust in automation - in particular the impact of mis-aligned trust, i.e. mistrust vs overreliance, the resulting impact on operator situation awareness (SA), the implications for takeover control from machine to human, and the role of experience and training on using automated transport systems. The results revealed several areas where experiences from the aviation and road domain can be transferable to other sectors. Experiences from maritime and rail transport, although limited, tend to confirm the general patterns. Remarkably, in the road sector where higher levels of automation are only recently introduced, there are clearer and more quantitative approaches to human factors, while other sectors focus only on mental modes. Other sectors could use similar approaches to define their own context-specific metrics. The paper makes a synthesis of key messages on automation safety in different transport sectors, and presents an assessment of their transferability.

Moving Into the Loop: An Investigation of Drivers' Steering Behavior in Highly Automated Vehicles

OBJECTIVE

This paper investigates driver engagement with vehicle automation and the transition to manual control in the context of a phenomenon that we have termed vicarious steering-drivers steering when the vehicle is under automated control.

BACKGROUND

Automated vehicles introduce many challenges, including disengagement from the driving task and out-of-the-loop performance decrement. We examine drivers' steering behavior when the automation is engaged, and steering input has no effect on the vehicle state. Such vicarious steering is a potential indicator of engagement for evaluating automated vehicles.

METHOD

A total of 32 female and 32 male drivers between 25 and 55 years of age participated in this experiment. A 2 × 2 between-subject design combined control algorithms and instructed responsibility. The control algorithms (lane centering and adaptive) were intended to convey the capability of the automation. The adaptive algorithm drifted across the lane center when latent hazards were present. The instructed levels of responsibility (driver primarily responsible and automation primarily responsible) were intended to replicate the admonitions of owners' manuals.

RESULTS

The adaptive algorithm increased vicarious steering (p < .001), but instructed responsibility did not (p = .67), and there was no interaction between the algorithm and the responsibility (p = .75). Vicarious steering was associated with an increase in transitions to manual control and glances to the road but was negatively associated with driving performance immediately after the transition to manual control.

CONCLUSION

Vicarious steering is a promising indicator of driver engagement when the vehicle is under automated control and automation algorithms can promote engagement.

Mode Awareness and Automated Driving—What Is It and How Can It Be Measured?

In SAE (Society of Automotive Engineers) Level 2, the driver has to monitor the traffic situation and system performance at all times, whereas the system assumes responsibility within a certain operational design domain in SAE Level 3. The different responsibility allocation in these automation modes requires the driver to always be aware of the currently active system and its limits to ensure a safe drive. For that reason, current research focuses on identifying factors that might promote mode awareness. There is, however, no gold standard for measuring mode awareness and different approaches are used to assess this highly complex construct. This circumstance complicates the comparability and validity of study results. We thus propose a measurement method that combines the knowledge and the behavior pillar of mode awareness. The latter is represented by the relational attention ratio in manual, Level 2 and Level 3 driving as well as the controllability of a system limit in Level 2. The knowledge aspect of mode awareness is operationalized by a questionnaire on the mental model for the automation systems after an initial instruction as well as an extensive enquiry following the driving sequence. Further assessments of system trust, engagement in non-driving related tasks and subjective mode awareness are proposed.

Driving automation state-of-mind: Using training to instigate rapid mental model development

The automotive industry is chugging along towards full autonomy, with a yet unknown time of arrival. The next call, however, is partial driving automation. At this interim station lurks many dangers, there-among them issues surrounding the partial performance of the driving task. Despite their potential for increased safety, these systems come with many inherent limitations and caveats, and their safe use depend on drivers correctly understanding their new role. Training is proposed as a potentially effective method of introducing drivers to the central aspects in this human-automation interaction. A proof-of-concept training program designed to introduce drivers to a partial automation system was developed. The effects of training were then evaluated through a between-group mixed-methods simulator experiment. Results indicate that trained drivers both self-report and exhibit an improved understanding of the automation system. They also report a significantly higher inclination to retake control in critical situation, than do their untrained counterparts.

The Sense of Agency in Driving Automation

Driving automation has been developing rapidly during the latest decade. However, all current technologies of driving automation still require human drivers’ monitoring and intervention. This means that during driving automation, the control by human driver and by the driving automation system are blended. In this case, if the human driver loses the sense of agency over the vehicle, he/she may not be able to actively engage in driving, and may excessively rely on the driving automation system. This review focuses on the subjective feeling of agency of the human driver over the vehicle in such situations. We address the possible measures of agency in driving automation, and discuss the insights from literatures on the sense of agency in joint control, robotics, automation, and driving assistance. We suggest that maintaining the sense of agency for human driver is important for ethical and safety reasons. We further propose a number of avenues for further research, which may help to better design an optimized driving automation considering human sense of agency.

Patterns of Sequential Off-Road Glances Indicate Levels of Distraction in Automated Driving

The safety concerns linked to semi-automated driving – more automation, less driver engagement – could be resolved by real-time driver monitoring with mitigation strategies. To achieve this, this paper analyzed an on-road dataset of sequential off-road glance behaviors under different levels of distraction in an autonomous vehicle trial named CANdrive. Several metrics based on sequential off-road glances were proposed and examined in terms of their capacity of measuring the levels of distraction. These findings are useful for the development of high-resolution driver state monitoring to improve safety in the collaboration between human driver and semi-autonomous vehicle.

Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures

OBJECTIVE

This article provides a review of empirical studies of automated vehicle takeovers and driver modeling to identify influential factors and their impacts on takeover performance and suggest driver models that can capture them.

BACKGROUND

Significant safety issues remain in automated-to-manual transitions of vehicle control. Developing models and computer simulations of automated vehicle control transitions may help designers mitigate these issues, but only if accurate models are used. Selecting accurate models requires estimating the impact of factors that influence takeovers.

METHOD

Articles describing automated vehicle takeovers or driver modeling research were identified through a systematic approach. Inclusion criteria were used to identify relevant studies and models of braking, steering, and the complete takeover process for further review.

RESULTS

The reviewed studies on automated vehicle takeovers identified several factors that significantly influence takeover time and post-takeover control. Drivers were found to respond similarly between manual emergencies and automated takeovers, albeit with a delay. The findings suggest that existing braking and steering models for manual driving may be applicable to modeling automated vehicle takeovers.

CONCLUSION

Time budget, repeated exposure to takeovers, silent failures, and handheld secondary tasks significantly influence takeover time. These factors in addition to takeover request modality, driving environment, non-handheld secondary tasks, level of automation, trust, fatigue, and alcohol significantly impact post-takeover control. Models that capture these effects through evidence accumulation were identified as promising directions for future work.

APPLICATION

Stakeholders interested in driver behavior during automated vehicle takeovers may use this article to identify starting points for their work.

Adaptive automation: automatically (dis)engaging automation during visually distracted driving

BACKGROUND

Automated driving is often proposed as a solution to human errors. However, fully automated driving has not yet reached the point where it can be implemented in real traffic. This study focused on adaptively allocating steering control either to the driver or to an automated pilot based on momentary driver distraction measured from an eye tracker.

METHODS

Participants (N = 31) steered a simulated vehicle with a fixed speed, and at specific moments were required to perform a visual secondary task (i.e., changing a CD). Three conditions were tested: (1) Manual driving (Manual), in which participants steered themselves. (2) An automated backup (Backup) condition, consisting of manual steering except during periods of visual distraction, where the driver was backed up by automated steering. (3) A forced manual drive (Forced) condition, consisting of automated steering except during periods of visual distraction, where the driver was forced into manual steering. In all three conditions, the speed of the vehicle was automatically kept at 70 km/h throughout the drive.

RESULTS

The Backup condition showed a decrease in mean and maximum absolute lateral error compared to the Manual condition. The Backup condition also showed the lowest self-reported workload ratings and yielded a higher acceptance rating than the Forced condition. The Forced condition showed a higher maximum absolute lateral error than the Backup condition.

DISCUSSION

In conclusion, the Backup condition was well accepted, and significantly improved performance when compared to the Manual and Forced conditions. Future research could use a higher level of simulator fidelity and a higher-quality eye-tracker.

Enhancing Trust in Autonomous Vehicles through Intelligent User Interfaces That Mimic Human Behavior

Autonomous vehicles use sensors and artificial intelligence to drive themselves. Surveys indicate that people are fascinated by the idea of autonomous driving, but are hesitant to relinquish control of the vehicle. Lack of trust seems to be the core reason for these concerns. In order to address this, an intelligent agent approach was implemented, as it has been argued that human traits increase trust in interfaces. Where other approaches mainly use anthropomorphism to shape appearances, the current approach uses anthropomorphism to shape the interaction, applying Gricean maxims (i.e., guidelines for effective conversation). The contribution of this approach was tested in a simulator that employed both a graphical and a conversational user interface, which were rated on likability, perceived intelligence, trust, and anthropomorphism. Results show that the conversational interface was trusted, liked, and anthropomorphized more, and was perceived as more intelligent, than the graphical user interface. Additionally, an interface that was portrayed as more confident in making decisions scored higher on all four constructs than one that was portrayed as having low confidence. These results together indicate that equipping autonomous vehicles with interfaces that mimic human behavior may help increasing people’s trust in, and, consequently, their acceptance of them.