"Are we in this together?": embedding social identity detection in drones improves emergency coordination

Autonomous systems, such as drones, are critical for emergency mitigation, management, and recovery. They provide situational awareness and deliver communication services which effectively guide emergency responders' decision making. This combination of technology and people comprises a socio-technical system. Yet, focusing on the use of drone technology as a solely operational tool, underplays its potential to enhance coordination between the different agents involved in mass emergencies, both human and non-human. This paper proposes a new methodological approach that capitalizes on social identity principles to enable this coordination in an evacuation operation. In the proposed approach, an adaptive drone uses sensor data to infer the group membership of the survivors it encounters during the operation. A corpus of 200 interactions of survivors' talk during real-life emergencies was computationally classified as being indicative of a shared identity or personal/no identity. This classification model, then, informed a game-theoretic model of human-robot interactions. Bayesian Nash Equilibrium analysis determined the predicted behavior for the human agent and the strategy that the drone needs to adopt to help with survivor evacuation. Using linguistic and synthetic data, we show that the identity-adaptive architecture outperformed two non-adaptive architectures in the number of successful evacuations. The identity-adaptive drone can infer which victims are likely to be helped by survivors and where help from emergency teams is needed. This facilitates effective coordination and adaptive performance. This study shows decision-making can be an emergent capacity that arises from the interactions of both human and non-human agents in a socio-technical system.

Versatile articulated aerial robot DRAGON: Aerial manipulation and grasping by vectorable thrust control

Various state-of-the-art works have achieved aerial manipulation and grasping by attaching additional manipulator to aerial robots. However, such a coupled platform has limitations with respect to the interaction force and mobility. In this paper, we present the successful implementation of aerial manipulation and grasping by a novel articulated aerial robot called DRAGON, in which a vectorable rotor unit is embedded in each link. The key to performing stable manipulation and grasping in the air is the usage of rotor vectoring apparatus having two degrees-of-freedom. First, a comprehensive flight control methodology for aerial transformation using the vectorable thrust force is developed with the consideration of the dynamics of vectoring actuators. This proposed control method can suppress the oscillation due to the dynamics of vectoring actuators and also allow the integration with external and internal wrenches for object manipulation and grasping. Second, an online thrust-level planning method for bimanual object grasping using the two ends of this articulated model is presented. The proposed grasping style is unique in that the vectorable thrust force is used as the internal wrench instead of the joint torque. Finally, we show the experimental results of evaluation on the proposed control and planning methods for object manipulation and grasping.

Agilicious: Open-source and open-hardware agile quadrotor for vision-based flight

Autonomous, agile quadrotor flight raises fundamental challenges for robotics research in terms of perception, planning, learning, and control. A versatile and standardized platform is needed to accelerate research and let practitioners focus on the core problems. To this end, we present Agilicious, a codesigned hardware and software framework tailored to autonomous, agile quadrotor flight. It is completely open source and open hardware and supports both model-based and neural network-based controllers. Also, it provides high thrust-to-weight and torque-to-inertia ratios for agility, onboard vision sensors, graphics processing unit (GPU)-accelerated compute hardware for real-time perception and neural network inference, a real-time flight controller, and a versatile software stack. In contrast to existing frameworks, Agilicious offers a unique combination of flexible software stack and high-performance hardware. We compare Agilicious with prior works and demonstrate it on different agile tasks, using both model-based and neural network-based controllers. Our demonstrators include trajectory tracking at up to 5g and 70 kilometers per hour in a motion capture system, and vision-based acrobatic flight and obstacle avoidance in both structured and unstructured environments using solely onboard perception. Last, we demonstrate its use for hardware-in-the-loop simulation in virtual reality environments. Because of its versatility, we believe that Agilicious supports the next generation of scientific and industrial quadrotor research.

Designing Expandable-Structure Robots for Human-Robot Interaction

In this paper, we survey the emerging design space of expandable structures in robotics, with a focus on how such structures may improve human-robot interactions. We detail various implementation considerations for researchers seeking to integrate such structures in their own work and describe how expandable structures may lead to novel forms of interaction for a variety of different robots and applications, including structures that enable robots to alter their form to augment or gain entirely new capabilities, such as enhancing manipulation or navigation, structures that improve robot safety, structures that enable new forms of communication, and structures for robot swarms that enable the swarm to change shape both individually and collectively. To illustrate how these considerations may be operationalized, we also present three case studies from our own research in expandable structure robots, sharing our design process and our findings regarding how such structures enable robots to produce novel behaviors that may capture human attention, convey information, mimic emotion, and provide new types of dynamic affordances.

CNN-Based Dense Monocular Visual SLAM for Real-Time UAV Exploration in Emergency Conditions

Unmanned Aerial Vehicles (UAVs) for 3D indoor mapping applications are often equipped with bulky and expensive sensors, such as LIDAR (Light Detection and Ranging) or depth cameras. The same task could be also performed by inexpensive RGB cameras installed on light and small platforms that are more agile to move in confined spaces, such as during emergencies. However, this task is still challenging because of the absence of a GNSS (Global Navigation Satellite System) signal that limits the localization (and scaling) of the UAV. The reduced density of points in feature-based monocular SLAM (Simultaneous Localization and Mapping) then limits the completeness of the delivered maps. In this paper, the real-time capabilities of a commercial, inexpensive UAV (DJI Tello) for indoor mapping are investigated. The work aims to assess its suitability for quick mapping in emergency conditions to support First Responders (FR) during rescue operations in collapsed buildings. The proposed solution only uses images in input and integrates SLAM and CNN-based (Convolutional Neural Networks) Single Image Depth Estimation (SIDE) algorithms to densify and scale the data and to deliver a map of the environment suitable for real-time exploration. The implemented algorithms, the training strategy of the network, and the first tests on the main elements of the proposed methodology are reported in detail. The results achieved in real indoor environments are also presented, demonstrating performances that are compatible with FRs’ requirements to explore indoor volumes before entering the building.

Gait Analysis for a Tiltrotor: The Dynamic Invertible Gait

A conventional feedback-linearization-based controller, when applied to a tiltrotor (eight inputs), results in extensive changes in tilting angles, which are not expected in practice. To solve this problem, we introduce the novel concept of “UAV gait” to restrict the tilting angles. The gait plan was initially used to solve the control problems in quadruped (four-legged) robots. Applying this approach, accompanied by feedback linearization, to a tiltrotor may give rise to the well-known non-invertible problem in the decoupling matrix. In this study, we explored invertible gait in a tiltrotor, and applied feedback linearization to stabilize the attitude and the altitude. The conditions necessary to achieve a full-rank decoupling matrix were deduced and simplified to near-zero roll and zero pitch. This paper proposes several invertible gaits to conduct an attitude–altitude control test. The accepted gaits within the region of interest were visualized. The simulation was conducted in Simulink, MATLAB. The results show promising responses in stabilizing attitude and altitude.

Shape morphing mechanical metamaterials through reversible plasticity

Biological organisms such as the octopus can reconfigure their shape and properties to perform diverse tasks. However, soft machines struggle to achieve complex configurations, morph into shape to support loads, and go between multiple states reversibly. Here, we introduce a multifunctional shape-morphing material with reversible and rapid polymorphic reconfigurability. We couple elastomeric kirigami with an unconventional reversible plasticity mechanism in metal alloys to rapidly (<0.1 seconds) morph flat sheets into complex, load-bearing shapes, with reversibility and self-healing through phase change. This kirigami composite overcomes trade-offs in deformability and load-bearing capacity and eliminates power requirements to sustain reconfigured shapes. We demonstrate this material through integration with onboard control, motors, and power to create a soft robotic morphing drone, which autonomously transforms from a ground to air vehicle and an underwater morphing machine, which can be reversibly deployed to collect cargo.

A Review of Quadrotor Unmanned Aerial Vehicles: Applications, Architectural Design and Control Algorithms

Over the past decade, unmanned aerial vehicles (UAVs) have received a significant attention due to their diverse capabilities for non-combatant and military applications. The primary aim of this study is to unveil a clear categorization overview for more than a decade worth of substantial progress in UAVs. The paper will begin with a general overview of the advancements, followed by an up-to-date explanation of the different mechanical structures and technical elements that have been included. The paper will then explore and examine various vertical take-off and landing (VTOL) configurations, followed by expressing the dynamics, applicable simulation tools and control strategies for a Quadrotor. In conclusion to this review, the dynamic system presented will always face limitations such as internal and/or external disturbances. Hence, this can be minimised by the choice of introducing appropriate control techniques or mechanical enhancements.

Design and Implementation of Morphed Multi-Rotor Vehicles with Real-Time Obstacle Detection and Sensing System

Multirotor unmanned aerial vehicles (MUAVs) are becoming more prominent for diverse real-world applications due to their inherent hovering ability, swift manoeuvring and vertical take-off landing capabilities. Nonetheless, to be entirely applicable for various obstacle prone environments, the conventional MUAVs may not be able to change their configuration depending on the available space and perform designated missions. It necessitates the morphing phenomenon of MUAVS, wherein it can alter their geometric structure autonomously. This article presents the development of a morphed MUAV based on a simple rotary actuation mechanism capable of driving each arm’s smoothly and satisfying the necessary reduction in workspace volume to navigate in the obstacle prone regions. The mathematical modelling for the folding mechanism was formulated, and corresponding kinematic analysis was performed to understand the synchronous motion characteristics of the arms during the folding of arms. Experiments were conducted by precisely actuating the servo motors based on the proximity ultrasonic sensor data to avoid the obstacle for achieving effective morphing of MUAV. The flight tests were conducted to estimate the endurance and attain a change in morphology of MUAV from “X-Configuration” to “H-Configuration” with the four arms actuated synchronously without time delay.

Nonlinear Robust Control of a New Reconfigurable Unmanned Aerial Vehicle

In this paper, a nonlinear robust Fast Terminal Sliding Mode Controller (FTSMC) is designed to control and stabilize a new reconfigurable Unmanned Aerial Vehicle (UAV) in the presence of uncertain and variable parameters. The studied UAV is an over-actuated system due the number of actuator control inputs. It can modify the length and the angles between its four arms in different ways, which result an important variation in its Center of Gravity (CoG), inertia, and control matrix. The proposed FTSMC offers many advantages such as, reaching the desired states in a finite-time unlike the conventional sliding mode, robustness vis-a-vis uncertain and unknown parameters, fast convergence towards the sliding surface, high accuracy and reducing the chattering phenomena. Furthermore, the closed-loop stability of the this UAV is ensured by the Lyapunov theory. The eight actuators used to rotate and extend the UAV arms are controlled by simple Proportional Integral Derivative (PID) controllers. Lastly, the robustness and efficiency of the proposed controller are evaluated through a flight scenario, where the UAV geometric parameters are variable over time.

The Design of Prometheus: A Reconfigurable UAV for Subterranean Mine Inspection

The inspection of legacy mine workings is a difficult, time consuming, costly task, as traditional methods require multiple boreholes to be drilled to allow sensors to be placed in the voids. Discrete sampling of the void from static locations also means that full coverage of the area cannot be achieved and occluded areas and side tunnels may not be fully mapped. The aim of the Prometheus project is to develop an autonomous robotic solution that is able to inspect the mine workings from a single borehole. This paper presents the challenges of operating autonomous aerial vehicles in such an environment, as well as physically entering the void with an autonomous robot. The paper address how some of these challenges can be overcome with bespoke design and intelligent controllers. It details the design of a reconfigurable UAV that is able to be deployed through a 150 mm borehole and unfold to a tip-to-tip diameter of 780 mm, allowing it to carry a payload suitable for a full autonomous mission.

A Survey on Swarming With Micro Air Vehicles: Fundamental Challenges and Constraints

This work presents a review and discussion of the challenges that must be solved in order to successfully develop swarms of Micro Air Vehicles (MAVs) for real world operations. From the discussion, we extract constraints and links that relate the local level MAV capabilities to the global operations of the swarm. These should be taken into account when designing swarm behaviors in order to maximize the utility of the group. At the lowest level, each MAV should operate safely. Robustness is often hailed as a pillar of swarm robotics, and a minimum level of local reliability is needed for it to propagate to the global level. An MAV must be capable of autonomous navigation within an environment with sufficient trustworthiness before the system can be scaled up. Once the operations of the single MAV are sufficiently secured for a task, the subsequent challenge is to allow the MAVs to sense one another within a neighborhood of interest. Relative localization of neighbors is a fundamental part of self-organizing robotic systems, enabling behaviors ranging from basic relative collision avoidance to higher level coordination. This ability, at times taken for granted, also must be sufficiently reliable. Moreover, herein lies a constraint: the design choice of the relative localization sensor has a direct link to the behaviors that the swarm can (and should) perform. Vision-based systems, for instance, force MAVs to fly within the field of view of their camera. Range or communication-based solutions, alternatively, provide omni-directional relative localization, yet can be victim to unobservable conditions under certain flight behaviors, such as parallel flight, and require constant relative excitation. At the swarm level, the final outcome is thus intrinsically influenced by the on-board abilities and sensors of the individual. The real-world behavior and operations of an MAV swarm intrinsically follow in a bottom-up fashion as a result of the local level limitations in cognition, relative knowledge, communication, power, and safety. Taking these local limitations into account when designing a global swarm behavior is key in order to take full advantage of the system, enabling local limitations to become true strengths of the swarm.

UAV-Based Structural Damage Mapping: A Review

Structural disaster damage detection and characterization is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of unmanned aerial vehicles (UAVs) in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. This study provides a comprehensive review of how UAV-based damage mapping has evolved from providing simple descriptive overviews of a disaster science, to more sophisticated texture and segmentation-based approaches, and finally to studies using advanced deep learning approaches, as well as multi-temporal and multi-perspective imagery to provide comprehensive damage descriptions. The paper further reviews studies on the utility of the developed mapping strategies and image processing pipelines for first responders, focusing especially on outcomes of two recent European research projects, RECONASS (Reconstruction and Recovery Planning: Rapid and Continuously Updated Construction Damage, and Related Needs Assessment) and INACHUS (Technological and Methodological Solutions for Integrated Wide Area Situation Awareness and Survivor Localization to Support Search and Rescue Teams). Finally, recent and emerging developments are reviewed, such as recent improvements in machine learning, increasing mapping autonomy, damage mapping in interior, GPS-denied environments, the utility of UAVs for infrastructure mapping and maintenance, as well as the emergence of UAVs with robotic abilities.

High-Level Teleoperation System for Aerial Exploration of Indoor Environments

Exploration of challenging indoor environments is a demanding task. While automation with aerial robots seems a promising solution, fully autonomous systems still struggle with high-level cognitive tasks and intuitive decision making. To facilitate automation, we introduce a novel teleoperation system with an aerial telerobot that is capable of handling all demanding low-level tasks. Motivated by the typical structure of indoor environments, the system creates an interactive scene topology in real-time that reduces scene details and supports affordances. Thus, difficult high-level tasks can be effectively supervised by a human operator. To elaborate on the effectiveness of our system during a real-world exploration mission, we conducted a user study. Despite being limited by real-world constraints, results indicate that our system better supports operators with indoor exploration, compared to a baseline system with traditional joystick control.

Evaluation of a Baseline Controller for Autonomous “Figure-8” Flights of a Morphing Geometry Quadcopter: Flight Performance

This article describes the design, fabrication, and flight test evaluation of a morphing geometry quadcopter capable of changing its intersection angle in-flight. The experiments were conducted at the Aircraft Computational and Resource Aware Fault Tolerance (AirCRAFT) Lab, Parks College of Engineering, Aviation and Technology at Saint Louis University, St. Louis, MO. The flight test matrix included flights in a “Figure-8” trajectory in two different morphing configurations (21° and 27°), as well as the nominal geometry configuration, two different flight velocities (1.5 m/s and 2.5 m/s), two different number of waypoints, and in three planes—horizontal, inclined, and double inclined. All the experiments were conducted using standard, off-the-shelf flight controller (Pixhawk) and autopilot firmware. Simulations of the morphed geometry indicate a reduction in pitch damping (42% for 21° morphing and 57.3% for 27° morphing) and roll damping (63.5% for 21° morphing and 65% for 27° morphing). Flight tests also demonstrated that the dynamic stability in roll and pitch dynamics were reduced, but the quadcopter was still stable under morphed geometry conditions. Morphed geometry also has an effect on the flight performance—with a higher number of waypoints (30) and higher velocity (2.5 m/s), the roll dynamics performed better as compared to the lower waypoints and lower velocity condition. The yaw dynamics remained consistent through all the flight conditions, and were not significantly affected by asymmetrical morphing of the quadcopter geometry. We also determined that higher waypoint and flight velocity conditions led to a small performance improvement in tracking the desired trajectory as well.

Preface: Latest Developments, Methodologies, and Applications Based on UAV Platforms

The use of Unmanned Aerial Vehicles (UAV) has boomed in the last decade, making these flying platforms an instrument for everyday data acquisition in many applications such as 3D modeling [...]