Experience in System Design for Human-Robot Teaming in Urban Search and Rescue

How and When Can Robots Be Team Members? Three Decades of Research on Human–Robot Teams

Artificial intelligence and robotic technologies have grown in sophistication and reach. Accordingly, research into mixed human–robot teams that comprise both robots and humans has expanded as well, attracting the attention of researchers from different disciplines, such as organizational behavior, human–robot interaction, cognitive science, and robotics. With this systematic literature review, the authors seek to establish deeper insights into existing research and sharpen the definitions of relevant terms. With a close consideration of 150 studies published between 1990 and 2020 that investigate mixed human–robot teams, conceptually or empirically, this article provides both a systematic evaluation of extant research and propositions for further research.

A model of symbiomemesis: machine education and communication as pillars for human-autonomy symbiosis

Symbiosis is a physiological phenomenon where organisms of different species develop social interdependencies through partnerships. Artificial agents need mechanisms to build their capacity to develop symbiotic relationships. In this paper, we discuss two pillars for these mechanisms: machine education (ME) and bi-directional communication. ME is a new revolution in artificial intelligence (AI) which aims at structuring the learning journey of AI-enabled autonomous systems. In addition to the design of a systematic curriculum, ME embeds the body of knowledge necessary for the social integration of AI, such as ethics, moral values and trust, into the evolutionary design and learning of the AI. ME promises to equip AI with skills to be ready to develop logic-based symbiosis with humans and in a manner that leads to a trustworthy and effective steady-state through the mental interaction between humans and autonomy; a state we name symbiomemesis to differentiate it from ecological symbiosis. The second pillar, bi-directional communication as a discourse enables information to flow between the AI systems and humans. We combine machine education and communication theory as the two prerequisites for symbiosis of AI agents and present a formal computational model of symbiomemesis to enable symbiotic human-autonomy teaming. This article is part of the theme issue 'Towards symbiotic autonomous systems'.

Real-time recognition of team behaviors by multisensory graph-embedded robot learning

Awareness of team behaviors (e.g., individual activities and team intents) plays a critical role in human–robot teaming. Autonomous robots need to be aware of the overall intent of the team they are collaborating with in order to effectively aid their human peers or augment the team’s capabilities. Team intents encode the goal of the team, which cannot be simply identified from a collection of individual activities. Instead, teammate relationships must also be encoded for team intent recognition. In this article, we introduce a novel representation learning approach to recognizing team intent awareness in real-time, based upon both individual human activities and the relationship between human peers in the team. Our approach formulates the task of robot learning for team intent recognition as a joint regularized optimization problem, which encodes individual activities as latent variables and represents teammate relationships through graph embedding. In addition, we design a new algorithm to efficiently solve the formulated regularized optimization problem, which possesses a theoretical guarantee to converge to the optimal solution. To evaluate our approach’s performance on team intent recognition, we test our approach on a public benchmark group activity dataset and a multisensory underground search and rescue team behavior dataset newly collected from robots in an underground environment, as well as perform a proof-of-concept case study on a physical robot. The experimental results have demonstrated both the superior accuracy of our proposed approach and its suitability for real-time applications on mobile robots.

Multirobot Confidence and Behavior Modeling: An Evaluation of Semiautonomous Task Performance and Efficiency

There is considerable interest in multirobot systems capable of performing spatially distributed, hazardous, and complex tasks as a team leveraging the unique abilities of humans and automated machines working alongside each other. The limitations of human perception and cognition affect operators’ ability to integrate information from multiple mobile robots, switch between their spatial frames of reference, and divide attention among many sensory inputs and command outputs. Automation is necessary to help the operator manage increasing demands as the number of robots (and humans) scales up. However, more automation does not necessarily equate to better performance. A generalized robot confidence model was developed, which transforms key operator attention indicators to a robot confidence value for each robot to enable the robots’ adaptive behaviors. This model was implemented in a multirobot test platform with the operator commanding robot trajectories using a computer mouse and an eye tracker providing gaze data used to estimate dynamic operator attention. The human-attention-based robot confidence model dynamically adapted the behavior of individual robots in response to operator attention. The model was successfully evaluated to reveal evidence linking average robot confidence to multirobot search task performance and efficiency. The contributions of this work provide essential steps toward effective human operation of multiple unmanned vehicles to perform spatially distributed and hazardous tasks in complex environments for space exploration, defense, homeland security, search and rescue, and other real-world applications.

Mutual Shaping in Swarm Robotics: User Studies in Fire and Rescue, Storage Organization, and Bridge Inspection

Many real-world applications have been suggested in the swarm robotics literature. However, there is a general lack of understanding of what needs to be done for robot swarms to be useful and trusted by users in reality. This paper aims to investigate user perception of robot swarms in the workplace, and inform design principles for the deployment of future swarms in real-world applications. Three qualitative studies with a total of 37 participants were done across three sectors: fire and rescue, storage organization, and bridge inspection. Each study examined the users' perceptions using focus groups and interviews. In this paper, we describe our findings regarding: the current processes and tools used in these professions and their main challenges; attitudes toward robot swarms assisting them; and the requirements that would encourage them to use robot swarms. We found that there was a generally positive reaction to robot swarms for information gathering and automation of simple processes. Furthermore, a human in the loop is preferred when it comes to decision making. Recommendations to increase trust and acceptance are related to transparency, accountability, safety, reliability, ease of maintenance, and ease of use. Finally, we found that mutual shaping, a methodology to create a bidirectional relationship between users and technology developers to incorporate societal choices in all stages of research and development, is a valid approach to increase knowledge and acceptance of swarm robotics. This paper contributes to the creation of such a culture of mutual shaping between researchers and users, toward increasing the chances of a successful deployment of robot swarms in the physical realm.

System Transparency in Shared Autonomy: A Mini Review

What does transparency mean in a shared autonomy framework? Different ways of understanding system transparency in human-robot interaction can be found in the state of the art. In one of the most common interpretations of the term, transparency is the observability and predictability of the system behavior, the understanding of what the system is doing, why, and what it will do next. Since the main methods to improve this kind of transparency are based on interface design and training, transparency is usually considered a property of such interfaces, while natural language explanations are a popular way to achieve transparent interfaces. Mechanical transparency is the robot capacity to follow human movements without human-perceptible resistive forces. Transparency improves system performance, helping reduce human errors, and builds trust in the system. One of the principles of user-centered design is to keep the user aware of the state of the system: a transparent design is a user-centered design. This article presents a review of the definitions and methods to improve transparency for applications with different interaction requirements and autonomy degrees, in order to clarify the role of transparency in shared autonomy, as well as to identify research gaps and potential future developments.

Shared planning and control for mobile robots with integral haptic feedback

This paper presents a novel bilateral shared framework for online trajectory generation for mobile robots. The robot navigates along a dynamic path, represented as a B-spline, whose parameters are jointly controlled by a human supervisor and an autonomous algorithm. The human steers the reference (ideal) path by acting on the path parameters that are also affected, at the same time, by the autonomous algorithm to ensure: (i) collision avoidance, (ii) path regularity, and (iii) proximity to some points of interest. These goals are achieved by combining a gradient descent-like control action with an automatic algorithm that re-initializes the traveled path (replanning) in cluttered environments to mitigate the effects of local minima. The control actions of both the human and the autonomous algorithm are fused via a filter that preserves a set of local geometrical properties of the path to ease the tracking task of the mobile robot. The bilateral component of the interaction is implemented via a force feedback that accounts for both human and autonomous control actions along the whole path, thus providing information about the mismatch between the reference and traveled path in an integral sense. The proposed framework is validated by means of realistic simulations and actual experiments deploying a quadrotor unmanned aerial vehicle (UAV) supervised by a human operator acting via a force-feedback haptic interface. Finally, a user study is presented to validate the effectiveness of the proposed framework and the usefulness of the provided force cues.

A distributed control algorithm for area search by a multi-robot team

In this paper, we present a novel algorithm for exploring an unknown environment using a team of mobile robots. The suggested algorithm is a grid-based search method that utilizes a triangular pattern which covers an area so that exploring the whole area is guaranteed. The proposed algorithm consists of two stages. In the first stage, all the members of the team make a common triangular grid of which they are located on the vertices. In the second stage, they start exploring the area by moving between vertices of the grid. Furthermore, it is assumed that the communication range of the robots is limited, and the algorithm is based on the information of the nearest neighbours of the robots. Moreover, we apply a new mapping method employed by robots during the search operation. A mathematically rigorous proof of convergence with probability 1 of the algorithm is given. Moreover, our algorithm is implemented and simulated using a simulator of the real robots and environment and also tested via experiments with Adept Pioneer 3DX wheeled mobile robots.

Mind the gap: detection and traversability analysis of terrain gaps using LIDAR for safe robot navigation

Safe navigation of robotic vehicles is considered as a key pre-requisite of successful mission operations within highly adverse and unconstrained environments. While there has been extensive research in the perception of positive obstacles, little progress can be accredited to the field of negative obstacles. This paper hypostatizes an elaborative attempt to address the problem of negative obstacle detection and traversability analysis in the form of gaps by processing 3-dimensional range data. The domain of application concerns Urban Search and Rescue scenarios that reflect environments of increased complexity in terms of diverse terrain irregularities. To allow real-time performance and, in turn, timely prevention of unrecoverable robotic states, the proposed approach is based on the application of efficient image morphological operations for noise reduction and border following the detection and grouping of gaps. Furthermore, we reason about gap traversability, a concept that is novel within the field. Traversability assessments are based on features extracted through Principal Component Analysis by exploring the spatial distribution of the interior of the individual gaps or the orientation distribution of the corresponding contour. The proposed approach is evaluated within a realistic scenario of a tunnel car accident site and a challenging outdoor scenario. Using a contemporary Search and Rescue robot, we have performed extensive experiments under various parameter settings that allowed the robot to always detect the real gaps, and either optimally cross over those that were traversable or otherwise avoid them.