Benefits of Imperfect Conflict Resolution Advisory Aids for Future Air Traffic Control

Do concurrent task demands impact the benefit of automation transparency?

Automated decision aids typically improve decision-making, but incorrect advice risks automation misuse or disuse. We examined the novel question of whether increased automation transparency improves the accuracy of automation use under conditions with/without concurrent (non-automated assisted) task demands. Participants completed an uninhabited vehicle (UV) management task whereby they assigned the best UV to complete missions. Automation advised the best UV but was not always correct. Concurrent non-automated task demands decreased the accuracy of automation use, and increased decision time and perceived workload. With no concurrent task demands, increased transparency which provided more information on how the automation made decisions, improved the accuracy of automation use. With concurrent task demands, increased transparency led to higher trust ratings, faster decisions, and a bias towards agreeing with automation. These outcomes indicate increased reliance on highly transparent automation under conditions with concurrent task demands and have potential implications for human-automation teaming design.

Exploring the Peak-End Effects in Air Traffic Controllers' Mental Workload Ratings

OBJECTIVE

This study examined whether professional air traffic controllers (ATCos) were subject to peak-end effects in reporting their mental workload after performing an air traffic control task, and in predicting their mental workload in future scenarios.

BACKGROUND

In affective experience studies, people's evaluation of a period of experience is strongly influenced by the most intense (peak) point and the endpoint. However, whether the effects exist in mental workload evaluations made by professional operators is still not known.

METHOD

In Study 1, 20 ATCos performed air traffic control scenarios on high-fidelity radar simulators and reported their mental workload. We used a 2 (high peak, low peak) × 2 (high end, low end) within-subject design. In Study 2, another group of 43 ATCos completed a survey asking them to predict their mental workload given the same air traffic control scenarios.

RESULTS

In Study 1, ATCos reported higher mental workload after completing the high-peak and the high-end scenarios. In contrast, in Study 2, ATCos predicted the peak workload effect but not the end workload effect when asked to predict their experience in dealing with the same scenarios.

CONCLUSION

Peak and end effects exist in subjective mental workload evaluation, but experts only had meta-cognitive awareness of the peak effect, and not the end effect.

APPLICATION

Researchers and practitioners that use subjective workload estimates for work design decisions need to be aware of the potential impact of peak and end task demand effects on subjective mental workload ratings provided by expert operators.

A Digital Twin-Based Platform towards Intelligent Automation with Virtual Counterparts of Flight and Air Traffic Control Operations

Automation technologies have been deployed widely to boost the efficiency of production and operations, to trim the complicated process, and to reduce the human error involved. Nevertheless, aviation remains human-centred and requires collaboration between different parties. Given the lack of a collaborative decision-making training platform for air traffic operations in the industry, this study utilises the concept of cyber-physical systems (CPS) to formulate a system architecture for pilots and air traffic control officers training in collaborative decision making by linking and integrating the virtual counterparts of flights and air traffic control operations. Collaborative decision-making training and the corresponding intelligent automation aids could be realised and supported. A performance analysis via a flight task undertaken with different computational load settings was prepared to evaluate the platform’s latency and integrity. The latency is presented using its 95% confidence interval, and integrity is presented using the percentage of data loss during wireless transmission. The results demonstrated convincing performance and a promising system robustness in both domains.

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

OBJECTIVE

To examine evidence of sensitivity, predictiveness, and methodological concerns regarding direct, objective measures of situation awareness (SA).

BACKGROUND

The ability to objectively measure SA is important to the evaluation of user interfaces and displays, training programs, and automation initiatives, as well as for studies that seek to better understand SA in both individuals and teams. A number of methodological criticisms have been raised creating significant confusion in the research field.

METHOD

A meta-analysis of 243 studies was conducted to examine evidence of sensitivity and predictiveness, and to address methodological questions regarding Situation Awareness Global Assessment Technique (SAGAT), Situation Present Assessment Technique (SPAM), and their variants.

RESULTS

SAGAT and SPAM were found to be equally predictive of performance. SPAM (64%) and real-time probes (73%) were found to have significantly lower sensitivity in comparison to SAGAT (94%). While SAGAT was found not to be overly memory reliant nor intrusive into operator performance, SPAM resulted in problems with intrusiveness in 40% of the studies examined, as well as problems with speed-accuracy tradeoffs, sampling bias, and confounds with workload. Concerns about memory reliance, the utility of these measures for assessing Team SA, and other issues are also addressed.

CONCLUSION

SAGAT was found to be a highly sensitive, reliable, and predictive measure of SA that is useful across a wide variety of domains and experimental settings.

APPLICATION

Direct, objective SA measurement provides useful and diagnostic insights for research and design in a wide variety of domains and study objectives.

Effect of automation transparency in the management of multiple unmanned vehicles

Automated decision aids can undoubtedly benefit system performance, but have the potential to provide incorrect advice, creating the possibility for automation disuse or misuse. This paper examined the extent to which increased automation transparency could improve the accuracy of automation use in a simulation of unmanned vehicle control. Participants were required to assign the best unmanned vehicle to complete missions. An automated recommender system provided advice but was not always reliable. Three levels of automation transparency were manipulated between-participants. Providing transparency regarding the reasoning underlying automated recommendations improved the accuracy of automation use, with no cost to decision time or subjective workload. Higher transparency in the form of projected decision outcome visualizations led to faster decisions, but less accurate automation use and an automation bias. Implications for the design of transparent interfaces to improve human-autonomy teaming outcomes in time-pressured environments are discussed.

Nautical Collision Avoidance : The Cognitive Challenges of Balancing Safety, Efficiency, and Procedures

OBJECTIVE

Experimentally investigate maneuver decision preferences in navigating ships to avoid a collision. How is safety (collision avoidance) balanced against efficiency (deviation from path and delay) and rules of the road under conditions of both trajectory certainty and uncertainty.

BACKGROUND

Human decision error is a prominent factor in nautical collisions, but the multiple factors of geometry of collisions and role of uncertainty have been little studied in empirical human factors literature.

APPROACH

Eighty-seven Mechanical Turk participants performed in a lower fidelity ship control simulation, depicting ownship and a cargo ship hazard on collision or near-collision trajectories of various conflict geometries, while controlling heading and speed with the sluggish relative dynamics. Experiment 1 involved the hazard on a straight trajectory. In Experiment 2, the hazard could turn on unpredictable trials. Participants were rewarded for efficiency and penalized for collisions or close passes.

RESULTS

Participants made few collisions, but did so more often when on a collision path. They sometimes violated the instructed rules of the road by maneuvering in front of the hazard ship's path. They preferred speed control to heading control. Performance degraded in conditions of uncertainty.

CONCLUSION

Data reveal an understanding of maneuver decisions and conditions that affect the balance between safety and efficiency.

APPLICATION

The simulation and data highlight the degrading role of uncertainty and provide a foundation upon which more complex questions can be asked, asked of more trained navigators, and decision support tools examined.

Human Performance Analysis of Processes for Retrieving Beidou Satellite Navigation System During Breakdown

Satellite navigation systems provide continuous, timely, and accurate signals of location, speed, and time to users all over the world. Although the running of these systems has become highly automated, the human operator is still vital for its continued operation, especially when certain equipment failures occur. In this paper, we examined 180 incidents of one particular type of equipment failure and the whole recovery process as recorded in the log files from a ground control center of the Beidou satellite navigation system. We extracted the information, including the technical description of the failure, the time when the fault occurred, the full recovery time, and the demographic information of the team members on the shift responsible for responding to the failure. We then transformed these information into the cognitive complexity of the task, time of day, shift handover period, and team skill composition. Multiple regression analysis showed that task complexity and shift handover were key predictors of recovery time. Time of day also influenced the recovery time, during midnight to 4 a.m., operators made longer responses. We also found that the fault handling processes could be improved if the team's most adept member is more skillful at that role than in other teams. We discussed the theoretical and practical implication of this study.