Effects of tactile cueing on concurrent performance of military and robotics tasks in a simulated multitasking environment

Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information

This work examines how tactile cues, encoded with azimuth and distance information, compare with visual and speech cues on performance and mental workload in a target detection task. Two experiments are reported using a simulated environment in which targets were presented at varying azimuth and distance locations. In the first experiment, participants engaged targets both while stationary and while in motion using tactile, visual, or speech cues. A no cueing control was included. In the second multi-modal experiment, participants completed the same task using cue pairings. Performance metrics consisted of hits, misses due to non-detection, misses due to inaccurate engagement, false alarms, response time, navigation errors as well as subjective ratings of mental workload scores were also collected. Results demonstrate the superiority of tactile cues as a means to communicate target location information either as a single modality or when paired with the two other cue types.

Interplay of Gravicentric, Egocentric, and Visual Cues About the Vertical in the Control of Arm Movement Direction

Our perception of the vertical corresponds to the weighted sum of gravicentric, egocentric, and visual cues. Here we evaluate the interplay of those cues not for the perceived but rather for the motor vertical. Participants were asked to flip an omnidirectional switch down while their egocentric vertical was dissociated from their visual-gravicentric vertical. Responses were directed mid-between the two verticals; specifically, the data suggest that the relative weight of congruent visual-gravicentric cues averages 0.62, and correspondingly, the relative weight of egocentric cues averages 0.38. We conclude that the interplay of visual-gravicentric cues with egocentric cues is similar for the motor and for the perceived vertical. Unexpectedly, we observed a consistent dependence of the motor vertical on hand position, possibly mediated by hand orientation or by spatial selective attention.

Toward a tactile language for human-robot interaction: two studies of tacton learning and performance

OBJECTIVE

Two experiments were performed to investigate the feasibility for robot-to-human communication of a tactile language using a lexicon of standardized tactons (tactile icons) within a sentence.

BACKGROUND

Improvements in autonomous systems technology and a growing demand within military operations are spurring interest in communication via vibrotactile displays. Tactile communication may become an important element of human-robot interaction (HRI), but it requires the development of messaging capabilities approaching the communication power of the speech and visual signals used in the military.

METHOD

In Experiment 1 (N = 38), we trained participants to identify sets of directional, dynamic, and static tactons and tested performance and workload following training. In Experiment 2 (N = 76), we introduced an extended training procedure and tested participants' ability to correctly identify two-tacton phrases. We also investigated the impact of multitasking on performance and workload. Individual difference factors were assessed.

RESULTS

Experiment 1 showed that participants found dynamic and static tactons difficult to learn, but the enhanced training procedure in Experiment 2 produced competency in performance for all tacton categories. Participants in the latter study also performed well on two-tacton phrases and when multitasking. However, some deficits in performance and elevation of workload were observed. Spatial ability predicted some aspects of performance in both studies.

CONCLUSIONS

Participants may be trained to identify both single tactons and tacton phrases, demonstrating the feasibility of developing a tactile language for HRI.

APPLICATION

Tactile communication may be incorporated into multi-modal communication systems for HRI. It also has potential for human-human communication in challenging environments.

The influence of camouflage, obstruction, familiarity and spatial ability on target identification from an unmanned ground vehicle

The purpose of this study was to examine the effects of environmental and cognitive factors on the identification of targets from an unmanned ground vehicle (UGV). This was accomplished by manipulating obstruction, camouflage and familiarity of objects in the environment, while also measuring spatial ability. The effects of these variables on target identification were studied by measuring performance of participants that observed pre-recorded video from a 1:35 scaled military operations in urban terrain facility. Analyses indicated that a combination of camouflage and obstruction caused the most detrimental effects on performance, and that there were differences in the recognition of familiar and unfamiliar targets. Further analysis indicated that these detrimental effects could only be overcome with a combination of target familiarity and spatial ability. The findings highlight the degree to which environmental factors hinder performance and the need for a multidimensional approach for improving performance under these conditions. Areas in need of future research are also discussed.

PRACTITIONER SUMMARY

Cognitive theory is applied to the problem of perception from UGVs. Results from an experimental study indicate that a combination of camouflage and obstruction caused the most detrimental effects on performance, with differences in the recognition of both familiar and unfamiliar targets. Familiarity and spatial ability interacted to predict the performance.

Supervisory control of multiple robots in dynamic tasking environments

A military targeting environment was simulated to examine the effects of an intelligent route-planning agent RoboLeader, which could support dynamic robot re-tasking based on battlefield developments, on the performance of robotics operators. We manipulated the level of assistance (LOAs) provided by RoboLeader as well as the presence of a visualisation tool that provided feedback to the participants on their primary task (target encapsulation) performance. Results showed that the participants' primary task benefited from RoboLeader on all LOAs conditions compared to manual performance; however, visualisation had little effect. Frequent video gamers demonstrated significantly better situation awareness of the mission environment than did infrequent gamers. Those participants with higher spatial ability performed better on a secondary target detection task than did those with lower spatial ability. Finally, participants' workload assessments were significantly lower when they were assisted by RoboLeader than when they performed the target entrapment task manually. Practitioner Summary: This study demonstrated the utility of an intelligent agent for enhancing robotics operators' supervisory control performance as well as reducing their workload during a complex urban scenario involving moving targets. The results furthered the understanding of the interplay among level-of-autonomy, multitasking performance and individual differences in military tasking environments.

Supervisory control of multiple robots: effects of imperfect automation and individual differences

OBJECTIVE

A military multitasking environment was simulated to examine the effects of an intelligent agent, RoboLeader, on the performance of robotics operators.

BACKGROUND

The participants' task was to manage a team of ground robots with the assistance of RoboLeader, an intelligent agent capable of coordinating the robots and changing their routes on the basis of battlefield developments.

METHOD

In the first experiment, RoboLeader was perfectly reliable; in the second experiment, RoboLeader's recommendations were manipulated to be either false-alarm prone or miss prone, with a reliability level of either 60% or 90%. The visual density of the targeting environment was manipulated by the presence or absence of friendly soldiers.

RESULTS

RoboLeader, when perfectly reliable, was helpful in reducing the overall mission times.The type of RoboLeader imperfection (false-alarm vs. miss prone) affected operators' performance of tasks involving visual scanning (target detection, route editing, and situation awareness). There was a consistent effect of visual density (clutter of the visual scene) for multiple performance measures. Participants' attentional control and video gaming experience affected their overall multitasking performance. In both experiments, participants with greater spatial ability consistently outperformed their low-spatial-ability counterparts in tasks that required effective visual scanning.

CONCLUSION

Intelligent agents, such as RoboLeader, can benefit the overall human-robot teaming performance. However, the effects of type of agent unreliability, tasking requirements, and individual differences have complex effects on human-agent interaction.

APPLICATION

The current results will facilitate the implementation of robots in military settings and will provide useful data to designs of systems for multirobot control.

Supervisory Control of Robots Using RoboLeader

We investigated the effectiveness of RoboLeader, an intelligent agent that could help the human operator control a team of robots, for enhancing the overall human-robot teaming performance. We compared the operators' target detection performance in the 4-robot and 8-robot conditions. The results showed that the participants detected significantly fewer targets with 8 robots vs. 4 robots. Although there were no significance differences between the RoboLeader and Baseline (no RoboLeader) conditions for target detection, the Roboleader group reduced their mission completion times by approximately 13% compared to Baseline. Those participants with higher spatial ability detected more targets than did those with lower spatial ability. Participants experienced significantly higher workload with 8 robots compared to the 4-robot condition, and those with better attentional control reported lower workload than did those with poorer attentional control.

UAV-guided navigation for ground robot tele-operation in a military reconnaissance environment

A military reconnaissance environment was simulated to examine the performance of ground robotics operators who were instructed to utilise streaming video from an unmanned aerial vehicle (UAV) to navigate his/her ground robot to the locations of the targets. The effects of participants' spatial ability on their performance and workload were also investigated. Results showed that participants' overall performance (speed and accuracy) was better when she/he had access to images from larger UAVs with fixed orientations, compared with other UAV conditions (baseline- no UAV, micro air vehicle and UAV with orbiting views). Participants experienced the highest workload when the UAV was orbiting. Those individuals with higher spatial ability performed significantly better and reported less workload than those with lower spatial ability. The results of the current study will further understanding of ground robot operators' target search performance based on streaming video from UAVs. The results will also facilitate the implementation of ground/air robots in military environments and will be useful to the future military system design and training community.

Effects of imperfect automation and individual differences on concurrent performance of military and robotics tasks in a simulated multitasking environment

This study investigated the performance and workload of the combined position of gunner and robotics operator in a simulated military multitasking environment. Specifically, the study investigated how aided target recognition (AiTR) capabilities for the gunnery task with imperfect reliability (false-alarm-prone vs. miss-prone) might affect the concurrent robotics and communication tasks. Additionally, the study examined whether performance was affected by individual differences in spatial ability and attentional control. Results showed that when the robotics task was simply monitoring the video, participants had the best performance in their gunnery and communication tasks and the lowest perceived workload, compared with the other robotics tasking conditions. There was a strong interaction between the type of AiTR unreliability and participants' perceived attentional control. Overall, for participants with higher perceived attentional control, false-alarm-prone alerts were more detrimental; for low attentional control participants, conversely, miss-prone automation was more harmful. Low spatial ability participants preferred visual cueing and high spatial ability participants favoured tactile cueing. Potential applications of the findings include personnel selection for robotics operation, robotics user interface designs and training development. The present results will provide further understanding of the interplays among automation reliability, multitasking performance and individual differences in military tasking environments. These results will also facilitate the implementation of robots in military settings and will provide useful data to military system designs.

Individual differences in route-learning strategy and associated working memory resources

OBJECTIVE

The current investigation examined individual differences in route-learning strategies and their relative demands on visuospatial versus verbal working memory (WM) resources in virtual environments.

BACKGROUND

Learning new routes is a resource-demanding activity that must often be carried out in conjunction with other concurrent tasks. Virtual environments (VEs) are increasingly being used for training and research, pointing to the importance of determining the strategies people use to learn routes in these environments.

METHODS

Participants classified as having good or poor sense of direction (SOD) attempted to learn novel routes while concurrently performing either a verbal (articulatory suppression) or a visuospatial (tapping) WM interference task.

RESULTS

Different navigational strategies were observed in each SOD group. Individuals with poor SOD relied more heavily on verbal rather than visuospatial WM resources, as evidenced by greater disruption to route-learning performance from the articulatory suppression task relative to the tapping task. Conversely, individuals with good SOD exhibited more route-learning disruption from the tapping task, suggesting a greater reliance on visuospatial WM resources.

CONCLUSION

Individuals differ from one another in the strategies they use and the WM resources they tap--verbal or visuospatial--to learn routes in VEs. Self-report measures can be used as indices of such individual differences in navigational strategy use in VE tasks.

APPLICATION

Assessing SOD and associated WM resources have implications for targeted training for navigation in VEs and for the design of in-vehicle navigation systems.

Tactile and Multisensory Spatial Warning Signals for Drivers

The last few years have seen many exciting developments in the area of tactile and multisensory interface design. One of the most rapidly-moving practical application areas for these findings is in the development of warning signals and information displays for drivers. For instance, tactile displays can be used to awaken sleepy drivers, to capture the attention of distracted drivers, and even to present more complex information to drivers who may be visually-overloaded. This review highlights the most important potential costs and benefits associated with the use of tactile and multisensory information displays in a vehicular setting. Multisensory displays that are based on the latest cognitive neuroscience research findings can capture driver attention significantly more effective than their unimodal (i.e., tactile) counterparts. Multisensory displays can also be used to transmit information more efficiently, as well as to reduce driver workload. Finally, we highlight the key research questions currently awaiting further research, including questions such as: Are tactile warning signals really intuitive? Are there certain regions of the body (or in the space surrounding the body) where tactile/multisensory warning signals are particularly effective? To what extent is the spatial coincidence and temporal synchrony of the individual sensory signals critical to determining the effectiveness of multisensory displays? And, finally, how does the issue of compliance vs. reliance (or the 'cry wolf' phenomenon associated with the presentation of signals that are perceived as false alarms) influence the effectiveness of tactile and/or multisensory warning signals?