Supervisory control of multiple robots: effects of imperfect automation and individual differences

The Relevance of Attention Control, Not Working Memory, in Human Factors

OBJECTIVE

Discuss the human factors relevance of attention control (AC), a domain-general ability to regulate information processing functions in the service of goal-directed behavior.

BACKGROUND

Working memory (WM) measures appear as predictors in various applied psychology studies. However, measures of WM reflect a mixture of memory storage and controlled attention making it difficult to interpret the meaning of significant WM-task relations for human factors. In light of new research, complex task performance may be better predicted or explained with new measures of attention control rather than WM.

METHOD

We briefly review the topic of individual differences in abilities in Human Factors. Next, we focus on WM, how it is measured, and what can be inferred from significant WM-task relations.

RESULTS

The theoretical underpinnings of attention control as a high-level factor that affects complex thought and behavior make it useful in human factors, which often study performance in complex and dynamic task environments. To facilitate research on attention control in applied settings, we discuss a validated measure of attention control that predicts more variance in complex task performance than WM. In contrast to existing measures of WM or AC, our measures of attention control only require 3 minutes each (10 minutes total) and may be less culture-bound making them suitable for use in applied settings.

CONCLUSION

Explaining or predicting task performance relations with attention control rather than WM may have dramatically different implications for designing more specific, equitable task interfaces, or training.

APPLICATION

A highly efficient ability predictor can help researchers and practitioners better understand task requirements for human factors interventions or performance prediction.

Supporting detection of hostile intentions: automated assistance in a dynamic decision-making context

In a dynamic decision-making task simulating basic ship movements, participants attempted, through a series of actions, to elicit and identify which one of six other ships was exhibiting either of two hostile behaviors. A high-performing, although imperfect, automated attention aid was introduced. It visually highlighted the ship categorized by an algorithm as the most likely to be hostile. Half of participants also received automation transparency in the form of a statement about why the hostile ship was highlighted. Results indicated that while the aid's advice was often complied with and hence led to higher accuracy with a shorter response time, detection was still suboptimal. Additionally, transparency had limited impacts on all aspects of performance. Implications for detection of hostile intentions and the challenges of supporting dynamic decision making are discussed.

No Evidence That Working Memory Modulates the Plasticity of the Nociceptive System, as Measured by Secondary Mechanical Hypersensitivity

The effect of cognition on the plasticity of the nociceptive system remains controversial. In this study, we examined whether working memory can buffer against the development of secondary hypersensitivity. Thirty-five healthy women participated in 3 experimental conditions. In each condition, they underwent electrical stimulation of the skin for 2 minutes (middle-frequency electrical stimulation [MFS]), which induces secondary hypersensitivity. During MFS, participants executed either an individually tailored and rewarded n-back task (working memory condition), a rewarded reaction-time task (non-working memory condition), or no task at all (control condition). Before and after MFS, participants rated the self-reported intensity and unpleasantness of mechanical pinprick stimuli. Fear of MFS was also assessed. Heart rate variability was measured to examine potential differences between the 3 conditions and steady-state evoked potentials to the electrical stimulation were recorded to investigate differences in cortical responses. We report no significant difference in hypersensitivity between the 3 conditions. Moreover, engaging in the cognitive tasks did not affect the heart rate variability or the steady-state evoked potentials. Interestingly, higher fear of MFS predicted greater hypersensitivity. In conclusion, we found no evidence that working memory affects the plasticity of the nociceptive system, yet pain-related fear plays a role. PERSPECTIVE: This study shows that the execution of a cognitive task, irrespective of cognitive load or working memory, does not significantly modulate the development of secondary hypersensitivity, heart rate variability, or steady-state evoked potentials. However, higher pain-related fear seems to contribute to greater hypersensitivity.

Gaming experience predicts UAS operator performance and workload in simulated search and rescue missions

Unmanned aircraft systems (UAS) operator training and selection procedures are still being refined to effectively address challenges related to performance, workload, and stress in UAS operation. Research suggests that experience with commercial videogames may test skills relevant to modern UAS operation. This study investigated the ability of videogame experience to predict operator performance, workload, and stress. Forty-nine participants performed 9 trials of a simulated search and rescue mission. It was expected that participants who more frequently played videogames would report lower levels of distress and workload, higher task engagement, and better overall performance. Results showed that gaming experience was negatively correlated with subjective workload and positively correlated with multiple measures of performance. Furthermore, nearly all observed gender-related differences were not present when gaming experience was controlled for. These results have implications for the role of gaming experience in remotely operated systems operator recruitment, selection, and training. Practitioner summary: This study examined how gaming experience influences UAS operator success in simulated search and rescue missions. Participants reported on their experience playing videogames before completing multiple experimental trials on a desktop computer. Results indicated that experience playing videogames significantly impacted performance, workload, and stress.

Performance Influence Mechanism of Individuals’ Spatial Ability in Teleoperation

BACKGROUND: Teleoperation enables performing tasks in hazardous or inaccessible environments. The relationship between spatial ability and teleoperation performance has been extensively studied; however, relatively few studies have considered examples wherein the specific influence mechanism between the two is examined. This study aims to explore how individuals’ spatial ability affects teleoperation performance.METHODS: Forty subjects completed teleoperated expedition and escape tasks in a virtual unfamiliar environment according to the assigned requirements. After each expedition task, subjects’ mental model about the unfamiliar environment was evaluated. The escape task performance was measured in terms of path length, completion time, and the number of collisions. The impact of spatial ability on escape task performance wherein mental model as a mediator was examined. The Bootstrapping method was used to examine the hypothesis regarding the mediating role of mental model in the influence of spatial ability on teleoperation performance.RESULTS: Subjects with higher spatial ability exhibited significantly better mental models and had fewer collisions. In addition, subjects with better mental models had significantly shorter path lengths and spent marginally less time on escape. In general, the mental model appeared important for path length and completion time, but not collisions.CONCLUSIONS: The combined results of the two tasks preliminarily proved that spatial ability affected path length of the escape task through the mental model after the exploration task. The findings are expected to aid in astronaut selection and teleoperation training for space station missions.Pan D, Liu D, Tian Z, Zhang Y. Performance influence mechanism of individuals’ spatial ability in teleoperation. Aerosp Med Hum Perform. 2022; 93(10):717–724.

Human-Autonomy Teaming: A Review and Analysis of the Empirical Literature

OBJECTIVE

We define human-autonomy teaming and offer a synthesis of the existing empirical research on the topic. Specifically, we identify the research environments, dependent variables, themes representing the key findings, and critical future research directions.

BACKGROUND

Whereas a burgeoning literature on high-performance teamwork identifies the factors critical to success, much less is known about how human-autonomy teams (HATs) achieve success. Human-autonomy teamwork involves humans working interdependently toward a common goal along with autonomous agents. Autonomous agents involve a degree of self-government and self-directed behavior (agency), and autonomous agents take on a unique role or set of tasks and work interdependently with human team members to achieve a shared objective.

METHOD

We searched the literature on human-autonomy teaming. To meet our criteria for inclusion, the paper needed to involve empirical research and meet our definition of human-autonomy teaming. We found 76 articles that met our criteria for inclusion.

RESULTS

We report on research environments and we find that the key independent variables involve autonomous agent characteristics, team composition, task characteristics, human individual differences, training, and communication. We identify themes for each of these and discuss the future research needs.

CONCLUSION

There are areas where research findings are clear and consistent, but there are many opportunities for future research. Particularly important will be research that identifies mechanisms linking team input to team output variables.

The role of attention control in complex real-world tasks

Working memory capacity is an important psychological construct, and many real-world phenomena are strongly associated with individual differences in working memory functioning. Although working memory and attention are intertwined, several studies have recently shown that individual differences in the general ability to control attention is more strongly predictive of human behavior than working memory capacity. In this review, we argue that researchers would therefore generally be better suited to studying the role of attention control rather than memory-based abilities in explaining real-world behavior and performance in humans. The review begins with a discussion of relevant literature on the nature and measurement of both working memory capacity and attention control, including recent developments in the study of individual differences of attention control. We then selectively review existing literature on the role of both working memory and attention in various applied settings and explain, in each case, why a switch in emphasis to attention control is warranted. Topics covered include psychological testing, cognitive training, education, sports, police decision-making, human factors, and disorders within clinical psychology. The review concludes with general recommendations and best practices for researchers interested in conducting studies of individual differences in attention control.

Teammates Instead of Tools: The Impacts of Level of Autonomy on Mission Performance and Human–Agent Teaming Dynamics in Multi-Agent Distributed Teams

Human–agent teaming (HAT) is becoming more commonplace across industry, military, and consumer settings. Agents are becoming more advanced, more integrated, and more responsible for tasks previously assigned to humans. In addition, the dyadic human–agent teaming nature is evolving from a one–one pair to one–many, in which the human is working with numerous agents to accomplish a task. As capabilities become more advanced and humanlike, the best method for humans and agents to effectively coordinate is still unknown. Therefore, current research must start diverting focus from how many agents can a human manage to how can agents and humans work together effectively. Levels of autonomy (LOAs), or varying levels of responsibility given to the agents, implemented specifically in the decision-making process could potentially address some of the issues related to workload, stress, performance, and trust. This study sought to explore the effects of different LOAs on human–machine team coordination, performance, trust, and decision making in hand with assessments of operator workload and stress in a simulated multi-unmanned aircraft vehicle (UAV) intelligence surveillance and reconnaissance (ISR) task. The results of the study can be used to identify human factor roadblocks to effective HAT and provide guidance for future designs of HAT. Additionally, the unique impacts of LOA and autonomous decision making by agents on trust are explored.

Trust Measurement in Human-Autonomy Teams: Development of a Conceptual Toolkit

The rise in artificial intelligence capabilities in autonomy-enabled systems and robotics has pushed research to address the unique nature of human-autonomy team collaboration. The goal of these advanced technologies is to enable rapid decision making, enhance situation awareness, promote shared understanding, and improve team dynamics. Simultaneously, use of these technologies is expected to reduce risk to those who collaborate with these systems. Yet, for appropriate human- autonomy teaming to take place, especially as we move beyond dyadic partnerships, proper calibration of team trust is needed to effectively coordinate interactions during high-risk operations. But to meet this end, critical measures of team trust for this new dynamic of human-autonomy teams are needed. This paper seeks to expand on trust measurement principles and the foundation of human-autonomy teaming to propose a “toolkit” of novel methods that support the development, maintenance and calibration of trust in human-autonomy teams operating within uncertain, risky, and dynamic environments.

A Unifying Framework for Human-Agent Collaborative Systems-Part I: Element and Relation Analysis

The human-agent collaboration (HAC) is a prospective research topic whose great applications and future scenarios have attracted vast attention. In a broad sense, the HAC system (HACS) can be broken down into six elements: "Man," "Agents," "Goal," "Network," "Environment," and "Tasks." By merging these elements and building a relation graph, this article proposes a systematic analysis framework for HACS, and attempts to make a comprehensive analysis of these elements and their relationships. We coin the abbreviation "MAGNET" to name the framework by stringing together the initials of the above six terms. The framework provides novel insights into analyzing various HAC patterns and integrates different types of HACSs in a unifying way. The presentation of the HACS framework is divided into two parts. This article, part I, presents the systematic analysis framework. Part II proposes a normalized two-stage top-level design procedure for designing an HACS from the perspective of MAGNET.

Where Do You Think You're Going?

Route planning is a critical behavior for human-intelligent agent (H-IA) team mobility. The scientific community has made major advances in improving route planner optimality and speed. However, human factors, such as the ability to predict and understand teammates’ actions and goals, are necessary for trust development in H-IA teams. Trust is especially critical when agents’ behaviors do not match human team members’ expectations, or the human cannot understand the agent's underlying reasoning process. To address this issue, the artificial intelligence community has pushed toward creating human-like agent behaviors using machine learning. The problem with this approach is that we do not yet have a clear understanding of what constitutes human-like behavior across the breadth of tasks that H-IA teams undertake. This article describes an investigation and comparison of human and agent route planning behaviors, the interplay between humans and agents in collaborative planning, and the role of trust in this collaborative process. Finally, we propose a data-driven methodology for characterizing and visualizing differences among routes planned by humans and agents. This methodology provides a means to advance compatible mental model metrics and theory by informing targeted transparency manipulations, thereby improving the speed and quality of routes produced by H-IA teams.

Exploring the Trust Influencing Mechanism of Robo-Advisor Service: A Mixed Method Approach

As a typical application of fintech, the robo-advisor has increasingly gained attention over the last decade. However, most research regarding the robo-advisor has focused on its development issues such as performance improvement and regulation, while limited research has paid attention to trust. This research extends the literature by investigating the trust influencing mechanism of robo-advisors by a mixed method approach. Specifically, we identified six salient trust influencing factors by qualitative interviews and proposed the research model based on trust transfer theory. This model was tested via a survey of 230 investors. Our study finds the significant influencing role of supervisory control and validates the relationships among trust influencing factors, trust in technologies, trust in vendor and trust in robo-advisor. Moreover, several differences between junior investors and senior investors are also found in our research. This study examines trust transfer theory in the new context of the robo-advisor and contributes to further development of this increasingly utilized service.

Effects of Touch, Voice, and Multimodal Input, and Task Load on Multiple-UAV Monitoring Performance During Simulated Manned-Unmanned Teaming in a Military Helicopter

OBJECTIVE

We evaluated three interface input methods for a simulated manned-unmanned teaming (MUM-T) supervisory control system designed for Air Mission Commanders (AMCs) in Black Hawk helicopters.

BACKGROUND

A key component of the U.S. Army's vision for unmanned aerial vehicles (UAVs) is to integrate UAVs into manned missions, called MUM-T (Department of Defense, 2010). One application of MUM-T is to provide the AMC of a team of Black Hawk helicopters control of multiple UAVs, offering advanced reconnaissance and real-time intelligence of flight routes and landing zones.

METHOD

Participants supervised a (simulated) team of two helicopters and three UAVs while traveling toward a landing zone to deploy ground troops. Participants classified aerial photographs collected by UAVs, monitored instrument warnings, and responded to radio communications. We manipulated interface input modality (touch, voice, multimodal) and task load (number of photographs).

RESULTS

Compared with voice, touch and multimodal control resulted in better performance on all tasks and resulted in lower subjective workload and greater subjective situation awareness, ps < .05. Participants with higher spatial ability classified more aerial photographs ( r = .75) and exhibited shorter response times to instrument warnings ( r = -.58) than participants with lower spatial ability.

CONCLUSION

Touchscreen and multimodal control were superior to voice control in a supervisory control task that involved monitoring visual displays and communicating on radio channels.

APPLICATION

Although voice control is often considered a more natural and less physically demanding input method, caution is needed when designing visual displays for users sharing common communication channels.

Too much or too little? Investigating the usability of high and low data displays of the same electronic medical record

The high data density on electronic medical record screens is touted as a major usability issue. However, it may not be a problem if the data is relevant and well-organized. Our objective was to test this assumption using a comprehensive set of measures that assess the three pillars of usability: efficiency (both physical and cognitive), effectiveness, and satisfaction. Physicians were asked to go through a series of tasks using two versions of the same electronic medical record: one where all the display items were separated into tabs (the original display), and one where important display items were grouped logically in one tab (the redesigned display). Results supported the hypothesis that combining relevant data in organized fashion into a smaller location would improve usability. The findings highlight the role of good display organization to mitigate the effects of high data density, as well as the importance of assessing cognitive load as part of usability studies.

Human-automation interaction for multiple robot control: the effect of varying automation assistance and individual differences on operator performance

In a human-automation interaction study, automation assistance level (AL) was investigated for its effects on operator performance in a dynamic, multi-tasking environment. Participants supervised a convoy of manned and unmanned vehicles traversing a simulated environment in three AL conditions, while maintaining situation awareness and identifying targets. Operators' situation awareness, target detection performance, workload and individual differences were evaluated. Results show increasing AL generally improved task performance and decreased perceived workload, however, differential effects due to operator spatial ability and perceived attentional control were found. Eye-tracking measures were useful in parsing out individual differences that subjective measures did not detect. At the highest AL, participants demonstrated potentially complacent behaviour, indicating task disengagement. Practitioner Summary: The effect of varying automation assistance level (AL) on operator performance on multiple tasks were examined in a within-subjects experiment. Findings indicated a moderate AL improved performance, while higher levels encouraged complacent behaviour. Effects due to individual differences suggest that effective AL depends on the underlying characteristics of the operator.

Voice Command Recognition in Multirobot Systems: Information Fusion

Recent Multirobot systems (MRS) are moving from theoretical considerations and from development and research centres to the area of practical applications. The solutions to real practical problems bring new challenges which are derived from actual requirements, while they are also interesting from theoretical point of view. One of the interesting areas of investigation is the problem of distributed data processing by limited computing and communication performance of individual components in MRS.

In this article, the authors try to demonstrate, using a simple example, the possibilities of distributed solution of classification tasks.

Such questions as:

– To what extent it is appropriate to distribute the tasks among individual elements of the system and to what extent to minimalize the requests on the communication subsystem? – Is it more appropriate, in the design concept of distributed data processing, to use a data fusion system, features fusion or decision fusion? are not universally solvable.

Therefore, we refrain from the analytical analysis and the choice of appropriate level of information fusion, but in four different scenarios we focus on solving ‘voice command recognition’ – we would like to show the advantages and disadvantages of individual approaches. The experiments described and the results achieved are based on simulation experiments and verified by experimental and demonstrative MRS.

A Framework to Guide the Assessment of Human-Machine Systems

OBJECTIVE

We have developed a framework for guiding measurement in human-machine systems.

BACKGROUND

The assessment of safety and performance in human-machine systems often relies on direct measurement, such as tracking reaction time and accidents. However, safety and performance emerge from the combination of several variables. The assessment of precursors to safety and performance are thus an important part of predicting and improving outcomes in human-machine systems.

METHOD

As part of an in-depth literature analysis involving peer-reviewed, empirical articles, we located and classified variables important to human-machine systems, giving a snapshot of the state of science on human-machine system safety and performance. Using this information, we created a framework of safety and performance in human-machine systems.

RESULTS

This framework details several inputs and processes that collectively influence safety and performance. Inputs are divided according to human, machine, and environmental inputs. Processes are divided into attitudes, behaviors, and cognitive variables. Each class of inputs influences the processes and, subsequently, outcomes that emerge in human-machine systems.

CONCLUSION

This framework offers a useful starting point for understanding the current state of the science and measuring many of the complex variables relating to safety and performance in human-machine systems.

APPLICATION

This framework can be applied to the design, development, and implementation of automated machines in spaceflight, military, and health care settings. We present a hypothetical example in our write-up of how it can be used to aid in project success.

The Effects of Information Level on Human-Agent Interaction for Route Planning

Two experiments were conducted to examine the effects of level of information on human operators’ route selection decisions. Experiment 1 examined how information about resource usage/requirements affected route selection decisions for a remotely based supervisor guiding a dismounted soldier unit through an urban environment. Experiment 2 increased the level of information from Experiment 1 by adding a robotic asset to the unit and providing its resource usage/requirements. Decision time increased as the level of information increased and increased again with the addition of the robotic asset. In addition, as the level of information increased, preference for specific information sources began to vary. In the condition with the greatest level of information available, participants displayed no clear consensus as to preferred information source, with many indicating they relied upon sources that were unsuitable for successful mission completion. Future research could investigate further into the complexity of appropriate display for user interfaces involving robotic assets.

Individual Differences in Multitasking Performance

The present study examined whether personality characteristics and general intelligence predict multitasking performance. The Multi-Attribute Task Battery-II was used to assess multitasking performance. Personality factors included the Big Five, openness, conscientiousness, extraversion, agreeableness, and neuroticism. The results indicated scores on general intelligence predict performance on the tracking task of the Multi-Attribute Task Battery-II, where higher scores of general intelligence predicted improved tracking performance. Additionally, conscientiousness and neuroticism were found to predict worsened performance on the resource management task of the Multi-Attribute Task Battery-II. Furthermore, agreeableness was found to predict perceived workload on the mental demand subscale of the Workload Rating Scale.

Intelligent Agent Transparency

We evaluated the usability and utility of an unmanned vehicle management interface that was developed based on the Situation awareness–based Agent Transparency model. We sought to examine the effect of increasing levels of agent transparency on operator task performance and perceived usability of the agent. Usability and utility were assessed through flash testing, a focus group, and experimental testing. While usability appeared to decrease with the portrayal of uncertainty, operator performance and reliance on key parts of the interface increased. Implications and next steps are discussed.

Experience of automation failures in training: effects on trust, automation bias, complacency and performance

This work examined the effects of operators' exposure to various types of automation failures in training. Forty-five participants were trained for 3.5 h on a simulated process control environment. During training, participants either experienced a fully reliable, automatic fault repair facility (i.e. faults detected and correctly diagnosed), a misdiagnosis-prone one (i.e. faults detected but not correctly diagnosed) or a miss-prone one (i.e. faults not detected). One week after training, participants were tested for 3 h, experiencing two types of automation failures (misdiagnosis, miss). The results showed that automation bias was very high when operators trained on miss-prone automation encountered a failure of the diagnostic system. Operator errors resulting from automation bias were much higher when automation misdiagnosed a fault than when it missed one. Differences in trust levels that were instilled by the different training experiences disappeared during the testing session. Practitioner Summary: The experience of automation failures during training has some consequences. A greater potential for operator errors may be expected when an automatic system failed to diagnose a fault than when it failed to detect one.

A Meta-Analysis of Factors Influencing the Development of Trust in Automation: Implications for Understanding Autonomy in Future Systems

OBJECTIVE

We used meta-analysis to assess research concerning human trust in automation to understand the foundation upon which future autonomous systems can be built.

BACKGROUND

Trust is increasingly important in the growing need for synergistic human-machine teaming. Thus, we expand on our previous meta-analytic foundation in the field of human-robot interaction to include all of automation interaction.

METHOD

We used meta-analysis to assess trust in automation. Thirty studies provided 164 pairwise effect sizes, and 16 studies provided 63 correlational effect sizes.

RESULTS

The overall effect size of all factors on trust development was ḡ = +0.48, and the correlational effect was [Formula: see text]  = +0.34, each of which represented medium effects. Moderator effects were observed for the human-related (ḡ  = +0.49; [Formula: see text] = +0.16) and automation-related (ḡ = +0.53; [Formula: see text] = +0.41) factors. Moderator effects specific to environmental factors proved insufficient in number to calculate at this time.

CONCLUSION

Findings provide a quantitative representation of factors influencing the development of trust in automation as well as identify additional areas of needed empirical research.

APPLICATION

This work has important implications to the enhancement of current and future human-automation interaction, especially in high-risk or extreme performance environments.

Effects of extended lay-off periods on performance and operator trust under adaptable automation

Little is known about the long-term effects of system reliability when operators do not use a system during an extended lay-off period. To examine threats to skill maintenance, 28 participants operated twice a simulation of a complex process control system for 2.5 h, with an 8-month retention interval between sessions. Operators were provided with an adaptable support system, which operated at one of the following reliability levels: 60%, 80% or 100%. Results showed that performance, workload, and trust remained stable at the second testing session, but operators lost self-confidence in their system management abilities. Finally, the effects of system reliability observed at the first testing session were largely found again at the second session. The findings overall suggest that adaptable automation may be a promising means to support operators in maintaining their performance at the second testing session.

Clutter in electronic medical records: examining its performance and attentional costs using eye tracking

OBJECTIVE

The objective was to use eye tracking to trace the underlying changes in attention allocation associated with the performance effects of clutter, stress, and task difficulty in visual search and noticing tasks.

BACKGROUND

Clutter can degrade performance in complex domains, yet more needs to be known about the associated changes in attention allocation, particularly in the presence of stress and for different tasks. Frequently used and relatively simple eye tracking metrics do not effectively capture the various effects of clutter, which is critical for comprehensively analyzing clutter and developing targeted, real-time countermeasures.

METHOD

Electronic medical records (EMRs) were chosen as the application domain for this research. Clutter, stress, and task difficulty were manipulated, and physicians' performance on search and noticing tasks was recorded. Several eye tracking metrics were used to trace attention allocation throughout those tasks, and subjective data were gathered via a debriefing questionnaire.

RESULTS

Clutter degraded performance in terms of response time and noticing accuracy. These decrements were largely accentuated by high stress and task difficulty. Eye tracking revealed the underlying attentional mechanisms, and several display-independent metrics were shown to be significant indicators of the effects of clutter.

CONCLUSION

Eye tracking provides a promising means to understand in detail (offline) and prevent (in real time) major performance breakdowns due to clutter.

APPLICATION

Display designers need to be aware of the risks of clutter in EMRs and other complex displays and can use the identified eye tracking metrics to evaluate and/or adjust their display.

Display clutter: a review of definitions and measurement techniques

OBJECTIVE

We aimed to synthesize the literature on display clutter by reviewing definitions and measurement techniques and to develop a comprehensive, ergonomics-oriented perspective on clutter. We provide guidance for the selection of measurement approaches that can capture the various aspects and effects of clutter on visual search in particular.

BACKGROUND

There is agreement that clutter may represent a significant problem to operators. The challenge is to determine the ideal middle ground between excessive data and insufficient information. However, definitions of clutter vary widely, which explains the range of measurement approaches, including image-processing algorithms, performance evaluation, subjective evaluation, and eye tracking. It is important to understand the affordances of each technique and provide guidance for their use.

METHOD

We provide a systematic review of clutter definitions and develop a performance-oriented perspective for ergonomics research. Next, we present a critical overview of clutter measurement approaches. The benefits and limitations of each technique are detailed, and recommendations for best practice are provided.

RESULTS

From an ergonomics perspective, clutter matters to the extent that it affects performance, particularly, visual search. A combination of measurement techniques can be employed in order to assess the performance costs stemming from the multiple aspects of clutter.

CONCLUSION

Display clutter is a multifaceted construct that is a problem when it results in performance and attentional costs. The assessment of these costs is critical and requires the use of appropriate and complementary techniques.

APPLICATION

It is important that the performance costs of clutter are detected reliably to be able to better support attention management.

Individual Differences in Human-Agent Teaming

Studies investigating the effects of level of autonomy (LOA) on workload and performance in human-agent teams typically utilize subjective measures, but do not often incorporate physiological measures. This paper examines how well eye movement data collected in a recent experiment converges with the findings suggested by the subjective measures. Several eye behavior measures (fixation count, average fixation duration, blink rate, saccade amplitude and pupil diameter) were evaluated, and findings based on these compared to findings of NASA-TLX and situation awareness questionnaires. In addition, individual differences due to perceived attentional control were evaluated. Findings indicate that the physiological measures account for variance in workload that typical subjective measures may not, as they are more sensitive to individual differences among participants.

Interactive effects of the COMT gene and training on individual differences in supervisory control of unmanned vehicles

OBJECTIVE

We examined whether a gene known to influence dopamine availability in the prefrontal cortex is associated with individual differences in learning a supervisory control task.

BACKGROUND

Methods are needed for selection and training of human operators who can effectively supervise multiple unmanned vehicles (UVs). Compared to the valine (Val) allele, the methionine (Met) allele of the COMT gene has been linked to superior executive function, but it is not known whether it is associated with training-related effects in multi-UV supervisory control performance.

METHOD

Ninety-nine healthy adults were genotyped for the COMT Val158Met single nucleotide polymorphism (rs4680) and divided into Met/Met, Val/Met, and Val/Val groups. Participants supervised six UVs in an air defense mission requiring them to attack incoming enemy aircraft and protect a no-fly zone from intruders in conditions of low and high task load (numbers of enemy aircraft). Training effects were examined across four blocks of trials in each task load condition.

RESULTS

Compared to the Val/Met and Val/Val groups, Met/Met individuals exhibited a greater increase in enemy targets destroyed and greater reduction in enemy red zone incursions across training blocks.

CONCLUSION

Individuals with the COMT Met/Met genotype can acquire skill in executive function tasks, such as multi-UV supervisory control, to a higher level and/or faster than other genotype groups.

APPLICATION

Potential applications of this research include the development of individualized training methods for operators of multi-UV systems and selecting personnel for complex supervisory control tasks.

Team performance in networked supervisory control of unmanned air vehicles: effects of automation, working memory, and communication content

OBJECTIVE

Assess team performance within a net-worked supervisory control setting while manipulating automated decision aids and monitoring team communication and working memory ability.

BACKGROUND

Networked systems such as multi-unmanned air vehicle (UAV) supervision have complex properties that make prediction of human-system performance difficult. Automated decision aid can provide valuable information to operators, individual abilities can limit or facilitate team performance, and team communication patterns can alter how effectively individuals work together. We hypothesized that reliable automation, higher working memory capacity, and increased communication rates of task-relevant information would offset performance decrements attributed to high task load.

METHOD

Two-person teams performed a simulated air defense task with two levels of task load and three levels of automated aid reliability. Teams communicated and received decision aid messages via chat window text messages.

RESULTS

Task Load x Automation effects were significant across all performance measures. Reliable automation limited the decline in team performance with increasing task load. Average team spatial working memory was a stronger predictor than other measures of team working memory. Frequency of team rapport and enemy location communications positively related to team performance, and word count was negatively related to team performance.

CONCLUSION

Reliable decision aiding mitigated team performance decline during increased task load during multi-UAV supervisory control. Team spatial working memory, communication of spatial information, and team rapport predicted team success.

APPLICATION

An automated decision aid can improve team performance under high task load. Assessment of spatial working memory and the communication of task-relevant information can help in operator and team selection in supervisory control systems.

Human Interaction With Multiple Remote Robots

In this chapter, I review research involving remote human supervision of multiple unmanned vehicles (UVs) using command complexity as an organizing construct. Multi-UV tasks range from foraging, requiring little coordination among UVs, to formation following, in which UVs must function as a cohesive unit. Command complexity, the degree to which operator effort increases with the number of supervised UVs, is used to categorize human interaction with multiple UVs. For systems in which each UV requires the same form of attention (O( n)), effort increases linearly with the number of UVs. For systems in which the control of one UV is dependent upon another (O(> n)), additional UVs impose greater than linear increases due to the expense of coordination. For other systems, an operator interacts with an autonomously coordinating group, and effort is unaffected by group size (O(1)). Studies of human/multi-UV interaction can be roughly grouped into O( n) supervision, involving one-to-one control of individual UVs, or O(1) commanding, in which higher-level commands are directed to a group. Research in O( n) command has centered on round-robin control, neglect tolerance, and attention switching. Approaches to O(1) command are divided into systems using autonomous path planning only, plan libraries, human-steered planners, and swarms. Each type of system has its advantages. Less complete work in scalable displays for multiple UVs is reviewed. Mixing levels of command is probably necessary to supervise multiple UVs performing realistic tasks. Research in O( n) control is mature and can provide quantitative and qualitative guidance for design. Interaction with planners and swarms is less mature but more critical to developing effective multi-UV systems capable of performing complex tasks.

Sensing, assessing, and augmenting threat detection: behavioral, neuroimaging, and brain stimulation evidence for the critical role of attention

Rapidly identifying the potentially threatening movements of other people and objects—biological motion perception and action understanding—is critical to maintaining security in many civilian and military settings. A key approach to improving threat detection in these environments is to sense when less than ideal conditions exist for the human observer, assess that condition relative to an expected standard, and if necessary use tools to augment human performance. Action perception is typically viewed as a relatively “primitive,” automatic function immune to top-down effects. However, recent research shows that attention is a top-down factor that has a critical influence on the identification of threat-related targets. In this paper we show that detection of motion-based threats is attention sensitive when surveillance images are obscured by other movements, when they are visually degraded, when other stimuli or tasks compete for attention, or when low-probability threats must be watched for over long periods of time—all features typical of operational security settings. Neuroimaging studies reveal that action understanding recruits a distributed network of brain regions, including the superior temporal cortex, intraparietal cortex, and inferior frontal cortex. Within this network, attention modulates activation of the superior temporal sulcus (STS) and middle temporal gyrus. The dorsal frontoparietal network may provide the source of attention-modulation signals to action representation areas. Stimulation of this attention network should therefore enhance threat detection. We show that transcranial Direct Current Stimulation (tDCS) at 2 mA accelerates perceptual learning of participants performing a challenging threat-detection task. Together, cognitive, neuroimaging, and brain stimulation studies provide converging evidence for the critical role of attention in the detection and understanding of threat-related intentional actions.

Adaptable and Adaptive Automation for Supervisory Control of Multiple Autonomous Vehicles

Supervisory control of multiple autonomous vehicles raises many issues concerning the balance of system autonomy with human interaction for optimal operator situation awareness and system performance. An unmanned vehicle simulation designed to manipulate the application of automation was used to evaluate participants’ performance on image analysis tasks under two automation control schemes: adaptable (level of automation directly manipulated by participant throughout trials) and adaptive (level of automation adapted as a function of participants’ performance on four types of tasks). The results showed that while adaptable automation increased workload, it also improved change detection, as well as operator confidence in task-related decision-making.

Supervisory control of multiple robots in dynamic tasking environments

A military targeting environment was simulated to examine the effects of an intelligent route-planning agent RoboLeader, which could support dynamic robot re-tasking based on battlefield developments, on the performance of robotics operators. We manipulated the level of assistance (LOAs) provided by RoboLeader as well as the presence of a visualisation tool that provided feedback to the participants on their primary task (target encapsulation) performance. Results showed that the participants' primary task benefited from RoboLeader on all LOAs conditions compared to manual performance; however, visualisation had little effect. Frequent video gamers demonstrated significantly better situation awareness of the mission environment than did infrequent gamers. Those participants with higher spatial ability performed better on a secondary target detection task than did those with lower spatial ability. Finally, participants' workload assessments were significantly lower when they were assisted by RoboLeader than when they performed the target entrapment task manually. Practitioner Summary: This study demonstrated the utility of an intelligent agent for enhancing robotics operators' supervisory control performance as well as reducing their workload during a complex urban scenario involving moving targets. The results furthered the understanding of the interplay among level-of-autonomy, multitasking performance and individual differences in military tasking environments.