Display signaling in augmented reality: effects of cue reliability and image realism on attention allocation and trust calibration

Beyond the Wizard of Oz: Negative Effects of Imperfect Machine Learning to Examine the Impact of Reliability of Augmented Reality Cues on Visual Search Performance

Despite knowing exactly what an object looks like, searching for it in a person's visual field is a time-consuming and error-prone experience. In Augmented Reality systems, new algorithms are proposed to speed up search time and reduce human errors. However, these algorithms might not always provide 100% accurate visual cues, which might affect users' perceived reliability of the algorithm and, thus, search performance. Here, we examined the detrimental effects of automation bias caused by imperfect cues presented in the Augmented Reality head-mounted display using the YOLOv5 machine learning model. 53 participants in the two groups received either 100% accurate visual cues or 88.9% accurate visual cues. Their performance was compared with the control condition, which did not include any additional cues. The results show how cueing may increase performance and shorten search times. The results also showed that performance with imperfect automation was much worse than perfect automation and that, consistent with automation bias, participants were frequently enticed by incorrect cues.

A Closer Look at Warning Cues on the Sustained Attention to Response Task Performance

OBJECTIVES

We investigated the effects of auditory cues of varying reliability levels on response inhibition performance using a target detection task to determine if external cues offer performance benefits. Further, we examined how the slope of the speed accuracy trade-off changes as a function of auditory cue reliability and used the trade-off to understand where any performance gains may be realized.

BACKGROUND

Researchers have proposed that the sustained attention to response task (SART) can be used to study the mechanisms causing failures of response inhibition. External cues may mitigate the results of motor inhibition failure. The extent to which external cues can effectively aid performance depends on the level of cue reliability.

METHOD

Ninety-one participants performed three SARTs with auditory cue assistance at three different levels of reliability (i.e. 0%, 60% and 100% reliable at cueing imminent No-Go stimuli).

RESULTS

We observed fewer errors of commission and faster reaction time in conditions with higher cue reliability. The slope of speed-accuracy trade-off relationship was impacted by cue reliability and was not a simple linear function.

CONCLUSION

Reliable auditory cues aid performance by reducing reaction time and errors of commission. Auditory cues also impact the relationship between speed and accuracy trade-off.

APPLICATION

Insights of cue effectiveness at different reliability levels help people make informed decisions in developing automation interfaces or sensors based on expected performance. Reliable cues mitigate the risk of impulsive errors; however, the reliability has to be high to have a noticeable impact on the speed-accuracy trade-off.

Is my AV crashing? An online photo-based experiment assessing whether shared intended pathway can help AV drivers anticipate silent failures

The shared responsibility between conditional AVs drivers demands shared understanding. Thus, a shared intended pathway (SIP)-a graphical display of the AV's planned manoeuvres in a head-up display to help drivers anticipate silent failures is proposed. An online, randomised photo experiment was conducted with 394 drivers in Australia. The photos presented traffic scenarios where the SIP forecast either safe or unsafe manoeuvres (silent failures). Participants were required to respond by selecting whether driver intervention was necessary or not. Additionally, the effects of presented object recognition bounding boxes which indicated whether a road user was recognised or not were also tested in the experiment. The SIP led to correct intervention choices 87% of the time, and to calibrating self-reported trust, perceived ease of use and usefulness. The bounding boxes found no significant effects. Results suggest SIPs can assist in monitoring conditional automation. Future research in simulator studies is recommended. Practitioner summary: Conditional AV drivers are expected to take-over control during failures. However, drivers are not informed about the AV's planned manoeuvres. A visual display that presents the shared intended pathway is proposed to help drivers mitigate silent failures. This online photo experiment found the display helped anticipate failures with 87% accuracy.

Evaluating Effective Connectivity of Trust in Human-Automation Interaction: A Dynamic Causal Modeling (DCM) Study

OBJECTIVE

Using dynamic causal modeling (DCM), we examined how credibility and reliability affected the way brain regions exert causal influence over each other-effective connectivity (EC)-in the context of trust in automation.

BACKGROUND

Multiple brain regions of the central executive network (CEN) and default mode network (DMN) have been implicated in trust judgment. However, the neural correlates of trust judgment are still relatively unexplored in terms of the directed information flow between brain regions.

METHOD

Sixteen participants observed the performance of four computer algorithms, which differed in credibility and reliability, of the system monitoring subtask of the Air Force Multi-Attribute Task Battery (AF-MATB). Using six brain regions of the CEN and DMN commonly identified to be activated in human trust, a total of 30 (forward, backward, and lateral) connection models were developed. Bayesian model averaging (BMA) was used to quantify the connectivity strength among the brain regions.

RESULTS

Relative to the high trust condition, low trust showed unique presence of specific connections, greater connectivity strengths from the prefrontal cortex, and greater network complexity. High trust condition showed no backward connections.

CONCLUSION

Results indicated that trust and distrust can be two distinctive neural processes in human-automation interaction-distrust being a more complex network than trust, possibly due to the increased cognitive load.

APPLICATION

The causal architecture of distributed brain regions inferred using DCM can help not only in the design of a balanced human-automation interface design but also in the proper use of automation in real-life situations.

Measuring the Efficiency of Automation-Aided Performance in a Simulated Baggage Screening Task

OBJECTIVE

The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making.

BACKGROUND

Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal.

METHOD

Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid's judgment relative to the raw stimuli, target certainty, and target prevalence.

RESULTS AND CONCLUSION

Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid's diagnosis with a probability of 50%. Performance was similar across experiments.

APPLICATION

Results suggest that human operators' performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.

Effects of automation reliability on error detection and attention to auditory stimuli in a multi-tasking environment

Automated aids are engineered to support operators' decision-making in complex and task-saturated environments, alerting them of system status and critical incidents. However, even the most advanced technologies are susceptible to failure. Monitoring imperfect automated systems poses unique challenges related to operator attention and workload. This study empirically examined the effect of unreliable automation on monitoring performance and responses to auditory stimuli in a multi-tasking environment. Participants completed an experimental trial consisting of four flight-related tasks while monitoring for automation failures at one of three levels of automation reliability. Participants in a high reliability condition responded more quickly and frequently to auditory messages. No performance differences were found in system monitoring performance between reliability conditions. These results are relevant to the design of automated system and delivery of automated alerts, and they have implications for operator attention allocation strategies in multi-tasking environments.

Interface Transparency Issues in Teleoperation

Transferring skills and expertise to remote places, without being present, is a new challenge for our digitally interconnected society. People can experience and perform actions in distant places through a robotic agent wearing immersive interfaces to feel physically there. However, technological contingencies can affect human perception, compromising skill-based performances. Considering the results from studies on human factors, a set of recommendations for the construction of immersive teleoperation systems is provided, followed by an example of the evaluation methodology. We developed a testbed to study perceptual issues that affect task performance while users manipulated the environment either through traditional or immersive interfaces. The analysis of its effect on perception, navigation, and manipulation relies on performances measures and subjective answers. The goal is to mitigate the effect of factors such as system latency, field of view, frame of reference, or frame rate to achieve the sense of telepresence. By decoupling the flows of an immersive teleoperation system, we aim to understand how vision and interaction fidelity affects spatial cognition. Results show that misalignments between the frame of reference for vision and motor-action or the use of tools affecting the sense of body position or movement have a higher effect on mental workload and spatial cognition.

Augmented reality-enhanced navigation in endoscopic sinus surgery: A prospective, randomized, controlled clinical trial

OBJECTIVE

Endoscopic sinus surgery represents the gold standard for surgical treatment of chronic sinus diseases. Thereby, navigation systems can be of distinct use. In our study, we tested the recently developed KARL STORZ NAV1 SinusTracker navigation software that incorporates elements of augmented reality (AR) to provide a better preoperative planning and guidance during the surgical procedure.

METHODS

One hundred patients with chronic sinus disease were operated on using either a conventional navigation software (n = 52, non-AR, control group) or a navigation software incorporating AR elements (n = 48, AR, intervention group). Incidence of postoperative complications, duration of surgery, surgeon-reported benefit from the navigation system and patient-reported postoperative rehabilitation were assessed.

RESULTS

The surgeons reported a higher benefit during surgery, used the navigation system for more surgical steps and spent longer time with preoperative image analysis when using the AR system as compared with the non-AR system. No significant differences were seen in terms of postoperative complications, target registration error, operation time and postoperative rehabilitation.

CONCLUSION

The AR enhanced navigation software shows a high acceptance by sinus surgeons in different stages of surgical training and offers potential benefits during surgery without affecting the duration of the operation or the incidence of postoperative complications.

LEVEL OF EVIDENCE

1b.

Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking

Objective

Surgical navigation is a well-established tool in endoscopic skull base surgery. However, navigational and endoscopic views are usually displayed on separate monitors, forcing the surgeon to focus on one or the other. Aiming to provide real-time integration of endoscopic and diagnostic imaging information, we present a new navigation technique based on augmented reality with fusion of intraoperative cone beam computed tomography (CBCT) on the endoscopic view. The aim of this study was to evaluate the accuracy of the method.

Material and methods

An augmented reality surgical navigation system (ARSN) with 3D CBCT capability was used. The navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. Intra-operative CBCT images were fused with the view of the surgical field obtained by the endoscope’s camera. Accuracy of CBCT image co-registration was tested using a custom-made grid with incorporated 3D spheres.

Results

Co-registration of the CBCT image on the endoscopic view was performed. Accuracy of the overlay, measured as mean target registration error (TRE), was 0.55 mm with a standard deviation of 0.24 mm and with a median value of 0.51mm and interquartile range of 0.39˗˗0.68 mm.

Conclusion

We present a novel augmented reality surgical navigation system, with fusion of intraoperative CBCT on the endoscopic view. The system shows sub-millimeter accuracy.

Extending Three Existing Models to Analysis of Trust in Automation: Signal Detection, Statistical Parameter Estimation, and Model-Based Control

OBJECTIVE

The objective is to propose three quantitative models of trust in automation.

BACKGROUND

Current trust-in-automation literature includes various definitions and frameworks, which are reviewed.

METHOD

This research shows how three existing models, namely those for signal detection, statistical parameter estimation calibration, and internal model-based control, can be revised and reinterpreted to apply to trust in automation useful for human-system interaction design.

RESULTS

The resulting reinterpretation is presented quantitatively and graphically, and the measures for trust and trust calibration are discussed, along with examples of application.

CONCLUSION

The resulting models can be applied to provide quantitative trust measures in future experiments or system designs.

APPLICATIONS

Simple examples are provided to explain how model application works for the three trust contexts that correspond to signal detection, parameter estimation calibration, and model-based open-loop control.

Understanding Human Over-Reliance On Technology

Implementing IT in medication-use systems reduces adverse drug events by decreasing human error. But over-reliance on technology can lead to automation bias and complacency.

Automation in visual inspection tasks: X-ray luggage screening supported by a system of direct, indirect or adaptable cueing with low and high system reliability

The present study evaluated three automation modes for improving performance in an X-ray luggage screening task. One hundred and forty participants were asked to detect the presence of prohibited items in X-ray images of cabin luggage. Twenty participants conducted this task without automatic support (control group), whereas the others worked with either indirect cues (system indicated the target presence without specifying its location), or direct cues (system pointed out the exact target location) or adaptable automation (participants could freely choose between no cue, direct and indirect cues). Furthermore, automatic support reliability was manipulated (low versus high). The results showed a clear advantage for direct cues regarding detection performance and response time. No benefits were observed for adaptable automation. Finally, high automation reliability led to better performance and higher operator trust. The findings overall confirmed that automatic support systems for luggage screening should be designed such that they provide direct, highly reliable cues.Practitioner summary: The present study confirmed previous findings showing better detection performance in X-ray images of luggage when supported by automation providing direct, highly reliable cues. Furthermore, participants used adaptable automation only to select their preferred level of automation. This behaviour did not provide the benefits expected under adaptable automation.

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

With the rise of increasingly complex artificial intelligence (AI), there is a need to design new methods to monitor AI in a transparent, human-aware manner. Decades of research have demonstrated that people, who are not aware of the exact performance levels of automated algorithms, often experience a mismatch in expectations. Consequently, they will often provide either too little or too much trust in an algorithm. Detecting such a mismatch in expectations, or trust calibration, remains a fundamental challenge in research investigating the use of automation. Due to the context-dependent nature of trust, universal measures of trust have not been established. Trust is a difficult construct to investigate because even the act of reflecting on how much a person trusts a certain agent can change the perception of that agent. We hypothesized that electroencephalograms (EEGs) would be able to provide such a universal index of trust without the need of self-report. In this work, EEGs were recorded for 21 participants (mean age = 22.1; 13 females) while they observed a series of algorithms perform a modified version of a flanker task. Each algorithm's degree of credibility and reliability were manipulated. We hypothesized that neural markers of action monitoring, such as the observational error-related negativity (oERN) and observational error positivity (oPe), are potential candidates for monitoring computer algorithm performance. Our findings demonstrate that (1) it is possible to reliably elicit both the oERN and oPe while participants monitored these computer algorithms, (2) the oPe, as opposed to the oERN, significantly distinguished between high and low reliability algorithms, and (3) the oPe significantly correlated with subjective measures of trust. This work provides the first evidence for the utility of neural correlates of error monitoring for examining trust in computer algorithms.

ISMP Medication Error Report Analysis: Understanding Human Over-reliance on Technology It's Exelan, Not Exelon Crash Cart Drug Mix-up Risk with Entering a "Test Order"

These medication errors have occurred in health care facilities at least once. They will happen again-perhaps where you work. Through education and alertness of personnel and procedural safeguards, they can be avoided. You should consider publishing accounts of errors in your newsletters and/or presenting them at your inservice training programs. Your assistance is required to continue this feature. The reports described here were received through the Institute for Safe Medication Practices (ISMP) Medication Errors Reporting Program. Any reports published by ISMP will be anonymous. Comments are also invited; the writers' names will be published if desired. ISMP may be contacted at the address shown below. Errors, close calls, or hazardous conditions may be reported directly to ISMP through the ISMP Web site (www.ismp.org), by calling 800-FAIL-SAFE, or via e-mail at ismpinfo@ismp.org. ISMP guarantees the confidentiality and security of the information received and respects reporters' wishes as to the level of detail included in publications.

Trust in haptic assistance: weighting visual and haptic cues based on error history

To effectively interpret and interact with the world, humans weight redundant estimates from different sensory cues to form one coherent, integrated estimate. Recent advancements in physical assistance systems, where guiding forces are computed by an intelligent agent, enable the presentation of augmented cues. It is unknown, however, if cue weighting can be extended to augmented cues. Previous research has shown that cue weighting is determined by the reliability (inversely related to uncertainty) of cues within a trial, yet augmented cues may also be affected by errors that vary over trials. In this study, we investigate whether people can learn to appropriately weight a haptic cue from an intelligent assistance system based on its error history. Subjects held a haptic device and reached to a hidden target using a visual (Gaussian distributed dots) and haptic (force channel) cue. The error of the augmented haptic cue varied from trial to trial based on a Gaussian distribution. Subjects learned to estimate the target location by weighting the visual and augmented haptic cues based on their perceptual uncertainty and experienced errors. With both cues available, subjects were able to find the target with an improved or equal performance compared to what was possible with one cue alone. Our results show that the brain can learn to reweight augmented cues from intelligent agents, akin to previous observations of the reweighting of naturally occurring cues. In addition, these results suggest that the weighting of a cue is not only affected by its within-trial reliability but also the history of errors.

Next-Generation Surgical Navigation Systems in Sinus and Skull Base Surgery

Over the past 25 years, rhinologists have adopted surgical navigation technology for endoscopic sinus and skull base procedures. Navigation systems often produce a wide target registration error (TRE). Ideally, next-generation systems will include a leap in target registration error reduce TRE through innovative hardware and software. Incorporation of microsensors will be another important innovation. Future systems are likely to include augmented reality, which can project overlays of critical anatomy on real-world endoscopic images. Recent trends in surgical navigation suggest a phase of rapid evolution.

Target Detection and Identification Performance Using an Automatic Target Detection System

OBJECTIVE

We investigated the effects of automatic target detection (ATD) on the detection and identification performance of soldiers.

BACKGROUND

Prior studies have shown that highlighting targets can aid their detection. We provided soldiers with ATD that was more likely to detect one target identity than another, potentially acting as an implicit identification aid.

METHOD

Twenty-eight soldiers detected and identified simulated human targets in an immersive virtual environment with and without ATD. Task difficulty was manipulated by varying scene illumination (day, night). The ATD identification bias was also manipulated (hostile bias, no bias, and friendly bias). We used signal detection measures to treat the identification results.

RESULTS

ATD presence improved detection performance, especially under high task difficulty (night illumination). Identification sensitivity was greater for cued than uncued targets. The identification decision criterion for cued targets varied with the ATD identification bias but showed a "sluggish beta" effect.

CONCLUSION

ATD helps soldiers detect and identify targets. The effects of biased ATD on identification should be considered with respect to the operational context.

APPLICATION

Less-than-perfectly-reliable ATD is a useful detection aid for dismounted soldiers. Disclosure of known ATD identification bias to the operator may aid the identification process.

Effects of the Advisor and Environment on Requesting and Complying With Automated Advice

Given the rapid technological advances in our society and the increase in artificial and automated advisors with whom we interact on a daily basis, it is becoming increasingly necessary to understand how users interact with and why they choose to request and follow advice from these types of advisors. More specifically, it is necessary to understand errors in advice utilization. In the present study, we propose a methodological framework for studying interactions between users and automated or other artificial advisors. Specifically, we propose the use of virtual environments and the tarp technique for stimulus sampling, ensuring sufficient sampling of important extreme values and the stimulus space between those extremes. We use this proposed framework to identify the impact of several factors on when and how advice is used. Additionally, because these interactions take place in different environments, we explore the impact of where the interaction takes place on the decision to interact. We varied the cost of advice, the reliability of the advisor, and the predictability of the environment to better understand the impact of these factors on the overutilization of suboptimal advisors and underutilization of optimal advisors. We found that less predictable environments, more reliable advisors, and lower costs for advice led to overutilization, whereas more predictable environments and less reliable advisors led to underutilization. Moreover, once advice was received, users took longer to make a final decision, suggesting less confidence and trust in the advisor when the reliability of the advisor was lower, the environment was less predictable, and the advice was not consistent with the environmental cues. These results contribute to a more complete understanding of advice utilization and trust in advisors.

Effects of Automation Failure in a Luggage Screening Task: A Comparison between Direct and Indirect Cueing

The present study investigated the use of two automated aids of different reliabilities in a luggage screening task. A Direct Cue consisting of a green circle around a potential target directs attention to a specific part of the luggage image, while an Indirect Cue, consisting of a green border around an image determined to have a target, does not. Direct Cues offer an advantage in visual inspection tasks because they guide attention to specific areas of the visual image but this can also cause attentional tunneling. Furthermore, the reliance on automation may negatively impact manual performance after the aid is removed or is no longer available. Thus, two issues were investigated in the current study: (1) how do failures in Direct and Indirect Cues affect reliance and (2) how does a complete failure affect performance after operators had the use of an automated aid? Results suggest that reliance patterns were more optimal with the Direct Cue than with the Indirect Cue and performance with a more reliable Indirect Cue was not much better than a less reliable one. The results also suggest that manual performance, when the aid was removed, was better for participants who had used the automated aids compared to control participants who did not have any use of the aid previously. The advantage of previously aided performance on subsequent manual performance was greatest for those who had used the more reliable Direct Cue. Explanations and implications are discussed.

Advanced Display Concepts for Uav Sensor Operations: Landmark Cues and Picture-in-Picture

UAV video imagery quality can be compromised by narrow field-of-view, environmental conditions, bandwidth limitations, or a highly cluttered scene. Advanced display concepts (e.g., synthetic vision) can potentially ameliorate video characteristics and enhance UAV operations. This study evaluated three display concepts for improving UAV sensor operator situation awareness: virtual flags overlaid on landmarks, a synthetically-generated imagery border (picture-in-picture), and a display-fixed spatial orientation symbol. Sixteen participants searched for ground landmarks using a UAV-mounted gimbaled camera simulation. Results indicated that the display concepts show promise for improving UAV operations. In particular, virtual flags reduced landmark search times by 40–58%. Although the picture-in-picture concept did not improve performance, comments suggest an alternative instantiation of the concept will improve utility.

Making Unreliable Automation Useful

Unreliable automation may be usefully applied to situation assessment tasks, but it requires careful analysis of task demands and good interface design. Here we analytically and empirically investigated the case of target detection in which more and less reliable automation identified promising search locations to guide users' attention. Participants searched a field of 44 locations to find four targets. Automation more or less reliably marked some locations as more promising than others. Participants could use even the less reliable automation to direct and facilitated their search compared with manual undirected search. A richer automated marking scheme proved useful but required a more complex search strategy that sometimes hindered participants.

Supporting Situation Assessment Through Attention Guidance: A Cost-Benefit and Depth of Processing Analysis

Automated support systems may be useful tools for aiding situation assessment in complex environments such as the military battlefield. These environments are marked by large amounts of information which often must be weighted and integrated into a meaningful judgment or assessment. The present research examines the effects of attention cueing on information integration tasks in static battlefield situations. Sixteen participants completed a resource allocation task for 56 battlefield scenarios (based on perceived threats). For half the trials, an automated system guided their attention to high-threat units. On 2 trials a memory probe was administered to assess the depth of processing of information, and on the final trial an automation failure was presented. Results demonstrated an overall allocation performance advantage for automation but poorer recall for automation-enhanced units. Half of the participants failed to attend to the system failure. Those participants who detected the failure were inferred to have processed the cues more deeply on the memory trials. The costs and benefits of automated cueing are discussed.

Battery electric vehicles - implications for the driver interface

The current study examines the human-machine interface of a battery electric vehicle (BEV) from a user-perspective, focussing on the evaluation of BEV-specific displays, the relevance of provided information and challenges for drivers due to the concept of electricity in a road vehicle. A sample of 40 users drove a BEV for 6 months. Data were gathered at three points of data collection. Participants perceived the BEV-specific displays as only moderately reliable and helpful for estimating the displayed parameters. This was even less the case after driving the BEV for 3 months. A taxonomy of user requirements was compiled revealing the need for improved and additional information, especially regarding energy consumption and efficiency. Drivers had difficulty understanding electrical units and the energy consumption of the BEV. On the background of general principles for display design, results provide implications how to display relevant information and how to facilitate drivers' understanding of energy consumption in BEVs. Practitioner Summary: Battery electric vehicle (BEV) displays need to incorporate new information. A taxonomy of user requirements was compiled revealing the need for improved and additional information in the BEV interface. Furthermore, drivers had trouble understanding electrical units and energy consumption; therefore, appropriate assistance is required. Design principles which are specifically important in the BEV context are discussed.

THE EFFECTS ON DRIVING SPEED OF A HEAD-UP DISPLAY OF ROAD WARNINGS (1)

This study investigated the superimposition of the projected markings on the road with head-up display, as well as their effects on the driver's speed. Two experiments were conducted. In Exp. 1, driving operations were simulated with a desktop computer to assess 18 deceleration markings (from the factors position, shape, and color) and determined the factors and levels influencing driving speeds. Based on the results of Exp. 1, six deceleration markings (from the factors shape and color) were selected in the driving simulator for conducting Exp. 2. The results of Exp. 1 showed that markings at the sides were better than the markings in the center. In Exp. 2, there was no significant difference between the effects of the arrangement of markings and the change of shape and color on driving stability. Yellow and white colors had no significant effect on speed; however, bar markings were better than zigzag markings. The results indicated that the projection of markings on a head-up display was helpful for indicating necessary deceleration.

Behavioral compliance for dynamic versus static signs in an immersive virtual environment

This study used an immersive virtual environment (IVE) to examine how dynamic features in signage affect behavioral compliance during a work-related task and an emergency egress. Ninety participants performed a work-related task followed by an emergency egress. Compliance with uncued and cued safety signs was assessed prior to an explosion/fire involving egress with exit signs. Although dynamic presentation produced the highest compliance, the difference between dynamic and static presentation was only statistically significant for uncued signs. Uncued signs, both static and dynamic, were effective in changing behavior compared to no/minimal signs. Findings are explained based on sign salience and on task differences. If signs must capture attention while individuals are attending to other tasks, salient (e.g., dynamic) signs are useful in benefiting compliance. This study demonstrates the potential for IVEs to serve as a useful tool in behavioral compliance research.

Augmented reality cues and elderly driver hazard perception

OBJECTIVE

The aim of this study was to evaluate the effectiveness of augmented reality (AR) cues in improving driving safety among elderly drivers who are at increased crash risk because of cognitive impairments.

BACKGROUND

Cognitively challenging driving environments pose a particular crash risk for elderly drivers. AR cuing is a promising technology to mitigate risk by directing driver attention to roadway hazards. We investigate whether AR cues improve or interfere with hazard perception in elderly drivers with age-related cognitive decline.

METHOD

A total of 20 elderly (M = 73 years, SD = 5) licensed drivers with a range of cognitive abilities measured by a speed-of-processing (SOP) composite participated in a 1-hr drive in an interactive, fixed-base driving simulator. Each participant drove through six straight, 6-mile-long, rural roadway scenarios following a lead vehicle. AR cues directed attention to potential roadside hazards in three of the scenarios, and the other three were uncued (baseline) drives. Effects of AR cuing were evaluated with respect to (a) detection of hazardous target objects, (b) interference with detecting nonhazardous secondary objects, and (c) impairment in maintaining safe distance behind a lead vehicle.

RESULTS

AR cuing improved the detection of hazardous target objects of low visibility. AR cues did not interfere with detection of nonhazardous secondary objects and did not impair ability to maintain safe distance behind a lead vehicle. SOP capacity did not moderate those effects.

CONCLUSION

AR cues show promise for improving elderly driver safety by increasing hazard detection likelihood without interfering with other driving tasks, such as maintaining safe headway.

Directing driver attention with augmented reality cues

This simulator study evaluated the effects of augmented reality (AR) cues designed to direct the attention of experienced drivers to roadside hazards. Twenty-seven healthy middle-aged licensed drivers with a range of attention capacity participated in a 54 mile (1.5 hour) drive in an interactive fixed-base driving simulator. Each participant received AR cues to potential roadside hazards in six simulated straight (9 mile long) rural roadway segments. Drivers were evaluated on response time for detecting a potentially hazardous event, detection accuracy for target (hazard) and non-target objects, and headway with respect to the hazards. Results showed no negative outcomes associated with interference. AR cues did not impair perception of non-target objects, including for drivers with lower attentional capacity. Results showed near significant response time benefits for AR cued hazards. AR cueing increased response rate for detecting pedestrians and warning signs but not vehicles. AR system false alarms and misses did not impair driver responses to potential hazards.

Surgeons blinded by enhanced navigation: the effect of augmented reality on attention

BACKGROUND

Advanced image-guidance systems allowing presentation of three-dimensional navigational data in real time are being developed enthusiastically for many medical procedures. Other industries, including aviation and the military, have noted that shifting attention toward such compelling assistance has detrimental effects. Using the detection rate of unexpected findings, we assess whether inattentional blindness is significant in a surgical context and evaluate the impact of on-screen navigational cuing with augmented reality.

METHODS

Surgeons and trainees performed an endoscopic navigation exercise on a cadaveric specimen. The subjects were randomized to either a standard endoscopic view (control) or an AR view consisting of an endoscopic video fused with anatomic contours. Two unexpected findings were presented in close proximity to the target point: one critical complication and one foreign body (screw). Task completion time, accuracy, and recognition of findings were recorded.

RESULTS

Detection of the complication was 0/15 in the AR group versus 7/17 in the control group (p = 0.008). Detection of the screw was 1/15 (AR) and 7/17 (control) (p = 0.041). Recognition of either finding was 12/17 for the control group and 1/15 for the AR group (p < 0.001). Accuracy was greater for the AR group than for the control group, with the median distance from the target point measuring respectively 2.10 mm (interquartile range [IQR], 1.29-2.37) and 4.13 (IQR, 3.11-7.39) (p < 0.001).

CONCLUSION

Inattentional blindness was evident in both groups. Although more accurate, the AR group was less likely to identify significant unexpected findings clearly within view. Advanced navigational displays may increase precision, but strategies to mitigate attentional costs need further investigation to allow safe implementation.

Automation bias: a systematic review of frequency, effect mediators, and mitigators

Automation bias (AB)--the tendency to over-rely on automation--has been studied in various academic fields. Clinical decision support systems (CDSS) aim to benefit the clinical decision-making process. Although most research shows overall improved performance with use, there is often a failure to recognize the new errors that CDSS can introduce. With a focus on healthcare, a systematic review of the literature from a variety of research fields has been carried out, assessing the frequency and severity of AB, the effect mediators, and interventions potentially mitigating this effect. This is discussed alongside automation-induced complacency, or insufficient monitoring of automation output. A mix of subject specific and freetext terms around the themes of automation, human-automation interaction, and task performance and error were used to search article databases. Of 13 821 retrieved papers, 74 met the inclusion criteria. User factors such as cognitive style, decision support systems (DSS), and task specific experience mediated AB, as did attitudinal driving factors such as trust and confidence. Environmental mediators included workload, task complexity, and time constraint, which pressurized cognitive resources. Mitigators of AB included implementation factors such as training and emphasizing user accountability, and DSS design factors such as the position of advice on the screen, updated confidence levels attached to DSS output, and the provision of information versus recommendation. By uncovering the mechanisms by which AB operates, this review aims to help optimize the clinical decision-making process for CDSS developers and healthcare practitioners.

Workload in Human-Robot Interaction: A Review of Manipulations and Outcomes

The current study reviews the relationship between manipulations of teleoperator workload and task outcomes, using multiple resource theory as the underlying framework. Results indicated that controlling more than two platforms is detrimental to many performance indices (reaction time, error rate), but overall productivity improves. For studies that manipulated workload for a single robot task, visual demands were a limiting factor, and interventions which reduced visual demands improved performance. The review concludes with guiding principles for managing workload and improving teleoperator performance.

Autonomy and Automation Reliability in Human-Robot Interaction: A Qualitative Review

The effectiveness and reliability of automation aids are critical topics in the area of human-robot interaction (HRI). As more tasks are subsumed by robots and autonomous systems, it is important to examine the relationships between these entities and their human operators. Research to date has covered various manipulations of autonomy, but this broad body of research is in need of focus and consistency. The current study presents a qualitative overview of research regarding levels and reliability of autonomy/control and the effects they have on important HRI-relevant outcome variables. Results indicate that autonomy and automation aids operate uniquely for different tasks, and that there are many complex factors that can affect not only performance but also usability, confidence, and safety. Unresolved issues in the field and challenges and opportunities for future research are also presented.

Binocular rivalry and head-worn displays

OBJECTIVE

We provide a review and analysis of much of the published literature on binocular rivalry that is relevant to the design and use of head-worn displays (HWDs).

BACKGROUND

This review draws heavily from both the basic vision literature and applied HWD literature in order to help provide insight for minimizing the effects of binocular rivalry when HWDs are worn.

METHOD

Included in this review are articles and books found cited in other works as well as articles and books obtained from an Internet search.

RESULTS

Issues discussed and summarized are (a) characteristics of binocular rivalry, (b) stimulus factors affecting rivalry, (c) cognitive variables affecting rivalry, and (d) tasks affected by rivalry.

CONCLUSION

This paper offers a set of recommendations for minimizing the effects of binocular rivalry when HWDs are used as well as recommendations for future research.

APPLICATION

Considerations of the basic vision literature on binocular rivalry will provide insight for future design solutions for HWDs.

Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information

A total of 24 pilots viewed dynamic encounters between their own aircraft and an intruder aircraft on a 2-D cockpit display of traffic information (CDTI) and estimated the point and time of closest approach. A three-level alerting system provided a correct categorical estimate of the projected miss distance on 83% of the trials. The remaining 17% of alerts were equally divided between misses and false alarms, of large and small magnitude. Roughly half the pilots depended on automation to improve estimation of miss distance relative to the baseline pilots, who viewed identical trials without the aid of automated alerts. Moreover, they did so more on the more difficult traffic trials resulting in improved performance on the 83% correct automation trials without causing harm on the 17% automation-error trials, compared to the baseline group. The automated alerts appeared to lead pilots to inspect the raw data more closely. While assisting the accurate prediction of miss distance, the automation led to an underestimate of the time remaining until the point of closest approach. The results point to the benefits of even imperfect automation in the strategic alerts characteristic of the CDTI, at least as long as this reliability remains high (above 80%).

Searching for signs, symbols, and icons: effects of time of day, visual complexity, and grouping

Searching for icons, symbols, or signs is an integral part of tasks involving computer or radar displays, head-up displays in aircraft, or attending to road traffic signs. Icons therefore need to be designed to optimize search times, taking into account the factors likely to slow down visual search. Three factors likely to adversely affect visual search were examined: the time of day at which search was carried out, the visual complexity of the icons, and the extent to which information features in the icon were grouped together. The speed with which participants searched icon arrays for a target was slower early in the afternoon, when icons were visually complex and when information features in icons were not grouped together to form a single object. Theories of attention that account for both feature-based and object-based search best explain these findings and are used to form the basis for ways of improving icon design.

Heuristic automation for decluttering tactical displays

Tactical displays can quickly become cluttered with large numbers of symbols that can compromise effective monitoring. Here, we studied how heuristic automation can aid users by intelligently "decluttering" the display. In a realistic simulated naval air defense task, 27 experienced U.S. Navy users monitored a cluttered airspace and executed defensive responses against significant threats. An algorithm continuously evaluated aircraft for their levels of threat and decluttered the less threatening ones by dimming their symbols. Users appropriately distrusted and spot-checked the automation's assessments, and decluttering had very little effect on which aircraft were judged as significantly threatening. Nonetheless, decluttering improved the timeliness of responses to threatening aircraft by 25% as compared with a baseline display with no decluttering; it was especially beneficial for threats in more peripheral locations, and 25 of 27 participants preferred decluttering. Heuristic automation, when properly designed to guide users' attention by decluttering less important objects, may prove valuable in many cluttered monitoring situations, including air traffic management, crisis team management, and tactical situation awareness in general.

Head up versus head down: the costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling

We conducted 2 experiments to investigate the clutter-scan trade-off between the cost of increasing clutter by overlaying complex information onto the forward field of view using a helmet-mounted display (HMD) and the cost of scanning when presenting this information on a handheld display. In the first experiment, this trade-off was examined in terms of the spatial accuracy of target cuing data in a relatively sparse display; in the second, the spatial accuracy of the cue was varied more radically in an information-rich display. Participants were asked to detect and identify targets hidden in the far domain while performing a monitoring task in the near domain using either an HMD or a handheld display. The results revealed that on a sparse display, the reduced scanning from the HMD presentation of cuing out-weighed the costs of clutter for cued targets, regardless of cue precision, but no benefit was found for uncued targets. When the HMD displayed task-irrelevant information, however, target detection was hindered by the extraneous clutter in the forward field of view relative to the handheld display condition, and this cost of clutter increased as the amount of data that needed to be inspected increased. Potential applications of this research include the development of design considerations for head-up displays for aviation and military applications.

Pilot maneuver choice and workload in free flight

Two experiments examined pilots' maneuver choice and visual workload in a free-flight simulation. In Experiment 1, 12 pilots flew a high-fidelity flight simulator with a cockpit display of traffic information and maneuvered to avoid traffic in a simulated free-flight environment. Pilots' choices reflected a preference to make vertical rather than lateral avoidance maneuvers and to climb rather than descend. Pilots avoided both complex maneuvers and airspeed maneuvers. The data were modeled in terms of how pilots traded off factors related to safety, efficiency, mental effort, and prior habits. In Experiment 2, 10 pilots flew the same maneuvers as the pilots in Experiment 1 but followed ATC instructions rather than using the CDTI. The CDTI in Experiment 1 occupied 25% of the pilots' visual attention. A comparison of scanning with Experiment 2 suggested that the CDTI pulled visual attention away from the outside world, but this attention diversion did not leave pilots vulnerable to missing traffic not annunciated on the CDTI. Actual or potential applications of the results include understanding the safety implications of presenting traffic displays in the cockpit, and the impact of pilot maneuver preferences on airspace procedures.